Usenet archaeology - Revision history

Jason Evans: /* archive.org */

2021-09-25T19:00:24Z

archive.org

← Older revision		Revision as of 19:00, 25 September 2021
Line 122:		Line 122:

	[[File:2-usenet-rec.png]]		[[File:2-usenet-rec.png]]

			This is the search result for "usenet-rec". From here you can go to the page for the '''REC''' Usenet hierarchy.

	[[File:3-show-all.png]]		[[File:3-show-all.png]]

Jason Evans: /* archive.org */

2021-09-25T18:59:02Z

archive.org

← Older revision		Revision as of 18:59, 25 September 2021
Line 118:		Line 118:

	[[File:1-search.png]]		[[File:1-search.png]]

			"Usenet-rec" is just a handy shortcut that I've found that takes me to the archive that I want. Replace "rec" with whatever hierarchy that you want.

	[[File:2-usenet-rec.png]]		[[File:2-usenet-rec.png]]

	[[File:3-show-all.png]]		[[File:3-show-all.png]]

Jason Evans: /* archive.org */

2021-09-25T18:57:53Z

archive.org

← Older revision		Revision as of 18:57, 25 September 2021
Line 115:		Line 115:
	In 2001, the website which had been the web's archive and public gateway to Usenet, Dejanews.com, was bought by Google and so Google Groups was born. A few years later in 2013, the [https://archive.org/details/usenethistorical Usenet Historical Collection] was created at archive.org with donations of newsgroup articles from Google.		In 2001, the website which had been the web's archive and public gateway to Usenet, Dejanews.com, was bought by Google and so Google Groups was born. A few years later in 2013, the [https://archive.org/details/usenethistorical Usenet Historical Collection] was created at archive.org with donations of newsgroup articles from Google.

	The first thing that we want to do when researching the files from archive.org is to download the files for that specific hierarchy. Let's first go to [https://archive.org] and look for the ~~rec~~ Usenet hierarchy.		The first thing that we want to do when researching the files from archive.org is to download the files for that specific hierarchy. Let's first go to https://archive.org and look for the '''REC''' Usenet hierarchy.

	[[File:1-search.png]]		[[File:1-search.png]]

			[[File:2-usenet-rec.png]]

			[[File:3-show-all.png]]

Jason Evans at 18:55, 25 September 2021

2021-09-25T18:55:26Z

← Older revision		Revision as of 18:55, 25 September 2021
Line 3:		Line 3:
	== Introduction: ==		== Introduction: ==

	I admin that Usenet Archaeology is a somewhat strange idea. The Usenet has been around since 1979. While that's only 41 years since the date of this tutorial, 41 years in terms of the Internet is prehistoric. The Usenet began as a project between Duke University and the University of North Carolina at Chapel Hill back in late 1979. The Internet as we know it has only been around since 1989 with the first internet service providers. How did articles get sent and received without the Internet? At first, news articles were sent only from machine to machine via modem. These modems were often very slow (300 bits per second) and long-distance telephone calls were very expensive. Storage was also expensive and so articles were often saved on magnetic tape if they were saved at all.		I admin that Usenet Archaeology is a somewhat strange idea. The Usenet has been around since 1979. While that's only 41 years since the date of this tutorial, 41 years in terms of the Internet is prehistoric. The Usenet began as a project between Duke University and the University of North Carolina at Chapel Hill back in late 1979. The Internet as we know it has only been around since 1989 with the first internet service providers. How did articles get sent and received without the Internet? At first, news articles were sent only from machine to machine via modem. These modems were often very slow (300 bits per second) and long-distance telephone calls were very expensive. Storage was also expensive and so articles were often saved on magnetic tape if they were saved at all.

			=== Note ===
			The instructions in this tutorial are multi-platform as much as possible so that multiple versions don't have to be written for different operating systems. It is also assumed that the reader has a basic understanding of how to use their operating system and file management such as how to de-compress files.

	== UTZOO Tapes: ==		== UTZOO Tapes: ==
Line 111:		Line 114:

	In 2001, the website which had been the web's archive and public gateway to Usenet, Dejanews.com, was bought by Google and so Google Groups was born. A few years later in 2013, the [https://archive.org/details/usenethistorical Usenet Historical Collection] was created at archive.org with donations of newsgroup articles from Google.		In 2001, the website which had been the web's archive and public gateway to Usenet, Dejanews.com, was bought by Google and so Google Groups was born. A few years later in 2013, the [https://archive.org/details/usenethistorical Usenet Historical Collection] was created at archive.org with donations of newsgroup articles from Google.

			The first thing that we want to do when researching the files from archive.org is to download the files for that specific hierarchy. Let's first go to [https://archive.org] and look for the rec Usenet hierarchy.

			[[File:1-search.png]]

Jason Evans: /* archive.org */

2021-09-25T18:47:41Z

archive.org

← Older revision		Revision as of 18:47, 25 September 2021
Line 110:		Line 110:
	== archive.org ==		== archive.org ==

	In 2001, the website which had been the web's archive and public gateway to Usenet, Dejanews.com, was bought by Google and so Google Groups was born. A few years later in 2013, the [https://archive.org/details/usenethistorical Usenet Historical Collection] was created at archive.org.		In 2001, the website which had been the web's archive and public gateway to Usenet, Dejanews.com, was bought by Google and so Google Groups was born. A few years later in 2013, the [https://archive.org/details/usenethistorical Usenet Historical Collection] was created at archive.org with donations of newsgroup articles from Google.

Jason Evans at 18:35, 25 September 2021

2021-09-25T18:35:59Z

← Older revision	Revision as of 18:35, 25 September 2021
Line 1:	Line 1:
		= Usenet Archaeology =

		== Introduction: ==

		I admin that Usenet Archaeology is a somewhat strange idea. The Usenet has been around since 1979. While that's only 41 years since the date of this tutorial, 41 years in terms of the Internet is prehistoric. The Usenet began as a project between Duke University and the University of North Carolina at Chapel Hill back in late 1979. The Internet as we know it has only been around since 1989 with the first internet service providers. How did articles get sent and received without the Internet? At first, news articles were sent only from machine to machine via modem. These modems were often very slow (300 bits per second) and long-distance telephone calls were very expensive. Storage was also expensive and so articles were often saved on magnetic tape if they were saved at all.

		== UTZOO Tapes: ==

		This brings us to the first trove of Usenet history, the UTZOO tapes. From [https://en.wikipedia.org/wiki/Henry_Spencer Wikipedia], "Between 1981 and 1991, while running the zoology department's computer system at the University of Toronto, [Henry] Spencer copied more than 2 million Usenet messages onto magnetic tapes. The 141 tapes wound up at the University of Western Ontario, where Google's Michael Schmidt tracked them down and, with the help of David Wiseman and others, got them transferred onto disks and into Google's archives." The story doesn't end there, unfortunately. The UTZOO tapes still exist in some for on Google Groups, but the original files on archive.org are no longer available. [https://archive.org/details/utzoo-wiseman-usenet-archive According to them], "In 2020 after sustained legal demands requesting a set of messages within the Usenet Archive be redacted, and to avoid further costs and accusations of manipulation should those demands be met, the archive has been removed from this URL and is not currently accessible to the public." With that said, they are still available online, but not where you might expect them in a repository like archive.org. A link is [http://annex.retroarchive.org/utzoo/index.html here]. If it is dead when you read it, just search online to try to find some other options.

		The UTZOO Tapes files are separated into files from the actual tapes. They begin with news001f1 and end with news141f1. The compressed "tarball" files are between 4 and 65MB. That may not sound like much, but that's actually over a million messages covering around a decade of Usenet history.

		Let's take a random sample of one of these files. I chose news086f1 at random. It appears that these articles are all from late 1988 to early 1989. Here is a sample breakdown of the directories in this file:

		<pre>
		.
		└── b107
		├── alt
		│ └── sources
		├── bionet
		│ ├── general
		│ ├── jobs
		│ ├── molbio
		│ │ ├── bio-matrix
		│ │ ├── evolution
		│ │ ├── genbank
		│ │ ├── methds-reagnts
		│ │ ├── news
		│ │ ├── proteins
		│ │ └── yeast
		│ └── sci-resources
		├── biz
		│ └── comp
		│ └── software
		├── can
		│ ├── general
		│ ├── jobs
		│ └── sun-stroke
		├── comp
		│ ├── ai
		│ │ ├── digest
		│ │ ├── neural-nets
		│ │ ├── nlang-know-rep
		│ │ └── vision

		</pre>

		If we want to go to biz.comp.software, we would go to the biz/comp/software subdirectory to read articles from that time. You might ask, isn't this stuff saved on Google groups? Yes and no. Let's take an article at random.

		/b107/news/misc/1832 in news086f1:

		<pre>
		Xref: utzoo news.admin:3599 news.misc:1832
		Path: utzoo!utgpu!water!watmath!clyde!att!osu-cis!tut.cis.ohio-state.edu!mailrus!bcm!watson!sob
		From: sob@watson.bcm.tmc.edu (Stan Barber)
		Newsgroups: news.admin,news.misc
		Subject: Re: Opinion on a "problem"
		Message-ID: <1301@gazette.bcm.tmc.edu>
		Date: 2 Oct 88 08:18:29 GMT
		References: <1169@nmtsun.nmt.edu>
		Sender: usenet@bcm.tmc.edu
		Reply-To: sob@watson.UUCP (Stan Barber)
		Organization: Baylor College of Medicine, Houston, TX
		Lines: 40

		In article <1169@nmtsun.nmt.edu> caasnsr@nmtsun.nmt.edu (Clifford Adams) writes:
		> I definitely favor KILL files for individuals, to "remove" a
		>troublesome person. KILLing a group of people based on a few is
		>counter to many of the dreams of USENET. Wasn't the Net supposed to
		>be the "great electronic anarchy", where everyone could speak, without
		>censorship or repression (at least in alt. groups)? A community where
		>all can share ideas, speaking the same language (ASCII), without
		>arbitrary differences like race or gender.
		>
		>--
		> Clifford A. Adams --- "I understand only inasmuch as I become."
		> ForthLisp Project Programmer (Goal: LISP interpreter in Forth)
		> caasnsr@nmtsun.nmt.edu ...cmcl2!lanl!unm-la!unmvax!nmtsun!caasnsr
		> (505) 835-6104 \| US Mail: Box 2439 Campus Station / Socorro, NM 87801


		I have always saw USENET as a great electronic community, not an anarchy.
		Anarchy implies no control. This is not the case. Newsadmins can impose
		whatever restrictions they choose. Some restrictions affect only the
		local site. Some can actually affect the net. Within this concept of
		community is the idea of citizenship. By this, I mean that there is
		implied responsibility for one's own actions. USENET has attempted to
		encourage everyone to use this resource responsibly. Some choose to abuse
		it. Others attempt to deal with this abuse either locally or by encouraging
		joint actions of a large group of newsadmins. Defining abuse and dealing
		with it are always sensitive issues, but they must be done for the
		community to continue to exist. Otherwise, more sites will exit this
		community for another.

		I don't disagree with your basic ideas regarding the alt-net. However,
		USENET remains a product of people and will continue to show all that is
		good and bad (racism and sexism) about people. To expect otherwise defeats
		the idea of USENET as a community. I applaude any who continue to encourage
		people to follow the guidelines that are defined. This encourges responsible
		usage of this resource. This resonsible use should be the hallmark of USENET.


		Stan internet: sob@bcm.tmc.edu Baylor College of Medicine
		Olan uucp: {rice,killer,hoptoad}!academ!sob
		Barber Opinions expressed are only mine.
		</pre>

		Now let's look at the same article in [https://groups.google.com/g/news.admin/c/izQLp2QBW7M/m/xy4dEAJb3dsJ Google Groups]. While the body of the article is the same, the headers are missing. Let's say you wanted to write a book on Usenet and decided to play it safe with copyright and try to contact the original authors. This is considerably more difficult if you only have the version of the article from Google Groups. Also, Google Groups lacks any kind of advanced "power" search functionality. If you are working on a Linux or BSD-based system, you have the power of tools like grep and bash which can make searching files like this much more convenient. Windows users can have this functionality also if they use tools like WSL or Cygwin. We'll talk more about Google Group later.

		== archive.org ==

		In 2001, the website which had been the web's archive and public gateway to Usenet, Dejanews.com, was bought by Google and so Google Groups was born. A few years later in 2013, the [https://archive.org/details/usenethistorical Usenet Historical Collection] was created at archive.org.

Jason Evans: Created blank page

2021-09-25T18:35:37Z

Created blank page

New page