What is a resource (or data) fork?
A Macintosh file has two parts: a data fork and a resource fork. Text files and GIF image files are examples of Macintosh files that are usually stored completely in the data fork, and have an empty (or nonexistent) resource fork. Applications, as a counter-example, store most if not all of their information in 'resources' in the resource fork and usually have an empty data fork.
Because this two-forked organization of files isn't very common, not only did Mac archive formats have to support them but a means to turn the two fork Mac file into a data fork had to be developed so that mac files could pass through non-macintosh machines (such as UNIX boxes, or MS-DOS machines) without being damaged.
This also means that without modification non-mac archives and encoding formats cannot be used to send mac files.
What is encoding?
To understand 'encoding' as the term is normally used on the Internet one needs to understand the difference between "binary" and ASCII. With the noted exception of text files computers store information in "binary" format which means that all 8-bits of a byte are used. By contrast ASCII originally only defined the first 7 bits of a byte setting the high bit in each byte to zero. As an added complication the character sets for byte values 128-255 used by ANSI and early (1981-c1990) IBM PCs differed.
As a result for 8-bit information to reliably be sent between computers it had to be translated into 7-bit ASCII text or 'encoded'. This was especially true of Usenet and e-mail which even today mostly supports 7-bit ASCII. Because 8-bits worth of data are being put into a 7-bit text file encoded files are always larger than their binary counterparts.
Due to its data and resource fork structure the Mac has an additional type of encoding structure: Binary encoding. Unlike ASCII encoding there is virturally no increase in file size but since these formats are 8-bit they cannot be used on their own in the remaining areas of the Internet that only support 7-bit (like E-mail and Usenet).
(a) What is BinHex? (b) What is uuencode? (c) What is Base64 ?
These are all ASCII encoding formats.
(a) BinHex 4.0, by Yves Lempereur, is a binary to text translator that can directly encode any Macintosh document (ie: it knows how to convert information in both the resource and data forks). Since the format is mainly used on already compressed files the RLE compression method that can be part of the format is rarely used. BinHex files can be easily recognized since they begin with the line: (This file must be converted with BinHex 4.0)
and are followed by a line starting with a colon, ':'. The BinHex encoding of the file follows, and is ended with another colon. Binhex 4.0 files also can be identified externally by the suffix ".hqx".
There is in fact a program called "BinHex 4.0" in various archives, but it has a bug wherein it refuses to decode .hqx files with very long names and you don't have to use it to convert files to and from BinHex.
It's best to use one of the other more powerful utilities like StuffIt Expander and other StuffIt programs, SunTar, and HQXer to name only a few. StuffIt Expander has the advantage of also being able to automatically expand StuffIt, Compact Pro, and Applelink archives and being available on PCs.
UNIX utilities that manipulate BinHex, MacBinary II, and other types of Macintosh files are also available.
The specifications to BinHex, should you be an interested programmer, are available at the University of Michigan's Macintosh archive site as mac/misc/documentation/binhex4.0specs.txt, or at InfoMac sites as dev/info/binhex-40-specs.txt.
There is also a program/format called "BinHex 5.0"; but it is NOT a more advanced version of "BinHex 4.0" but rather a separate _binary_ encoding format (see [2.2]). BinHex 5.0, written by Yves Lempereur, in 1985 was the first MacBinary converter available. BinHex 5.0 (also called MacBinary I) was replaced by the MacBinary II format which added support for several new MacOS features.
As new versions of BinHex were developed, they encoded only the new format but continued to decode all previous formats:
BinHex 1.0 encodes .hex and decodes .hex
BinHex 2.0 encodes .hex & .hcx and decodes .hex & .hcx
BinHex 3.0 never existed
BinHex 4.0 encodes .hqx and decodes .hex, .hcx & .hqx
BinHex 5.0 encodes MacBinary I and decodes .hex, .hcx, .hqx & MacBinary I
(b) "uuencode" is a binary to text translator that serves the same purpose as BinHex, except that it knows nothing about the Macintosh resource/data fork structure. Uuencode was designed to allow UNIX binary files to be easily transferred through text-only interfaces, such as e-mail. Every uuencoded file contains a line similar to:
begin 644 usa-map.gif followed by a series of lines of ASCII text characters (which are normally 60 characters long and begin with the letter 'M'). The file ends with a line containing the word 'end'. There may be other special keywords included. Externally uuencode files are usially denoted with the suffix ".uu" or ".uue".
Usually, one won't find Macintosh files in uuencode format; however, most non-Macintosh specific binary data posted to Usenet is uuencoded, so if you wish to use any of this data (such as the images posted in alt.binaries.* and elsewhere), you will need to deal with uuencode. The programs 'uuencode' and 'uudecode' exist on most UNIX systems. If not, don't worry as there are many programs allow you to convert to and from uuencode using your Macintosh.
(c) Base64 is the encoding format used by Multipurpose Internet Mail Extension (Mime) files. The reason mime uses Base64 rather than the more popular uuencode format is that uuencode is not really a standard but rather a collection of related but different formats. This rendered uuencode impractical as a cross platform encoding format.
Mac files are usially binary encoded (usially in AppleDouble) before being encoded in Base64.
a) What are AppleSingle and AppleDouble? b) What is MacBinary?
These are all Mac binary encoding formats.
AppleSingle and AppleDouble were developed out of a need to share Mac file between the MacOS and A/UX (Apple's first UnixOS) as well as allowing A/UX users to edit MacOS files. The specs of these
formats can be found at <http://www.cis.ohio-state.edu/htbin/rfc/rfc1740.html>.
AppleDouble is useful today because it devides a Mac file into two files: one for the data fork (with original filename) and the other for resource fork (with '%' prefixing the original filename) This made it easy to adopt AppleDouble to MIME - have non-mac systems simply ignore the '%' file.
Mac e-mail programs that use AppleSingle and AppleDouble encode them into Base64.
b) MacBinary is the Mac's standard binary encoding (see [2.2]) format. MacBinary's purpose is to encapsulate *all* information (including the filename, creation and modification dates, file type and creator) contained in a Macintosh file for transport over a non-Macintosh medium.
Although a Macintosh program (called MacBinary) does exist to do the converting to and from MacBinary, almost all modern Macintosh telecommunications and Internet programs have thecapability of converting and unconverting MacBinary files for you. ZTerm, for example, can be configured to automatically detect when a MacBinary file is being received and to convert this file to its original representation; or, if you are uploading, ZTerm can optionally encode the file into MacBinary before sending. Fetch, and most other shareware and commercial products have equivalent or similar capabilities.
Dennis Brothers, Yves Lempereur, and others gathered on CompuServe to discuss what eventually became the original MacBinary standard. According to Lempereur, "We finally agreed on using the MacTerminal format (without the modified XModem protocol). I then wrote BinHex 5.0 to support MacBinary. A year later, the same group got together on CompuServe again and created MacBinary II."
MacBinary I is the name given to the old MacBinary standard.
MacBinary II is the name given to the new MacBinary standard which everybody uses today; in common usage, MacBinary means
MacBinary II.
MacBinary III is an update to the vernerable c1987 format that supports the icon badge custom routing information finder flags that are part of MacOS 8.5.
Since then, BinHex and the MacBinary II have become the standard way of encapsulating Macintosh files for transfer over foreign systems throughout the Internet, USENET, and elsewhere. MacBinary is also used as a way to retain Mac file information in non-mac archive formats. For example ZipIt uses MacBinary in this manner for the PC .zip format.
MacBinary's correct MIME type is "application/x-macbinary" and if you want StuffIt Expander to launch when you double click on the file set the type and creator fields to BINA and SITx.
What do file suffixes like .hqx, .sit, .bin, etc ... mean and how can I convert such files back to normal Macintosh applications and documents?
Most files available by FTP or posted to Usenet are modified twice to allow them to more easily pass through foreign computer systems. First they are compressed and then either ASCII or Binary encoded with BinHex (.hqx) and MacBinary (.bin) being the formats of choice for Macintosh users and [2.4] for an explanation of these formats).
Generally the suffix on these files only tells you the encoding method used and nothing about the compression method. As a result StuffIt Expander has become the defacto decoder utility especially when combined with the StuffIt Engine. You can use the following table to determine what Macintosh programs handle which formats. For a more complete description of the various Macintosh archival programs, see the excellent FAQ for comp.sys.mac.apps.
This table is also part of the Mac-FTP-list and listed on its own as format-chart.txt both of which are at
<http://members.aol.com/BruceG6069/> as well as being archived on any info-mac mirror site, in the /info-mac/comm/ directory.
unix gzip .uu/ .b64/
Macintosh .cpt .sit .hqx .bin .arc .zip .tar .Z .gz/.z .uue .mime*
Stuffit Expander** D D D D D D
w/ Engine** D X X D D D D D D D D
ShrinkWrap*^ D D D D D D D D D D D
StuffIt Lite*** D X/4 X X
StuffIt Deluxe*** D X X X D D X X D X D
ArcMac X
Compact Pro 1.5.1 X D/N X
Decoder 1.3.7 D D
MacCompress X
MacGzip D X
MPack 1.5.1 D D X
SunTar 2.1.4 X X X X D
Tar 4.0b X
uucd 2.4.6 X D
ZipIt 1.3.8 D D X
Other unix gzip .uu/ .b64/
computers .cpt .sit .hqx .bin .arc .zip .tar .Z .gz/.z .uue .mime*
Aladdin Expander D D D D D D D
Aladdin DropStuff X X
binhex-pc-13 X
extrac.exe D
macutil (unix)
hexbin D D
macunpack D D/N D D D
mcvert (unix) X X
MPack D D X
PKZIP X
xferp110 (win) X X X
D = Decode only
N = Cannot decompress current Deluxe .sit [Type SITD] files
4 = Cannot handle Stuffit 5.0 format
X = Encode and decode
.sit refers to all versions of the Stuffit format. A '/' denotes the inability to handle certain formats as outlined in the legend above.
.hqx = BinHex4; .bin = BinHex5, MacBinary I, II, and III
* .b64/.mime (Base 64) refers to the encoding format used by the Multipurpose Internet Mail Extension. For more information consult the MIME FAQ.
<http://www.cis.ohio-state.edu/hypertext/faq/usenet/mail/mime-faq/top.html>
** Engine refers to the StuffIt Engine which is part of StuffIt Deluxe and DropStuff with Expander Enhancer [a $30 shareware addon for Stuffit Expander] Programs that can use the Stuffit Engine are marked with a *^.
Current public versions of the Expander and the Engine are 5.1 as of this writing.
Note - DSEE 4.5 is NOT compatable with SE 5.0.
*** Stuffit Deluxe 3.5-5.0 translators can be used with Stuffit Lite. Current versions are 5.0 and 3.6.0 respectively as of this writing.
Table 2.5.1
Note: .gz and .Z compression systems, while both native to UNIX, are completely different, and these suffixes cannot not be interchanged.
WARNING: .hqx, .uu, .b64, and .txt files are the ONLY files that can be downloaded in ASCII mode; all others must be downloaded in BINARY mode for the file to decompress properly. This is especially true of ".bin" and "unstuffed" files. Otherwise you will get errors like "unreadable file" or "file is corrupt" when you try to decompress them.
Less commonly used formats. Those followed by a + are Mac formats.
.arj
PC format common to European sites. Decompressed by unArjMac,
DeArj, and Stuffit Expander for Windows.
.dd +
Disk Doubler {Mac} format. Decompresses by DDExpand and DiskDoubler.
.exe
DOS/Windows executable file (program); also used to create self-extracting archives. An .exe file used as a self-extracting archive can usually be decompressed with Stuffit Expander w/ DSEE.
.html (.htm)
WWW document. Used by WWW browsers such as Netscape and lynx.
.image/.img/.ima/ (related format - .smi) +
These are all disk image extensions. They represent Mac disk image (.image/.img), Microsoft Disk Image Utility (.img), and Winimage (.ima) formats. Most can be mounted via StuffIt Expander 4.5 or
ShrinkWrap 3.0
<http://www.aladdinsys.com/developers/shrinkwrap/index.html>.
To eliminate the need for a mounter program there now exists a self mounting disk image format called .smi. For a history of Shrinkwrap consult the 2.1 site <http://www.halcyon.com/shrinkwrap/>.
Note that .img is also used as an graphic file extension and needs GraphicConverter to view.
.lzh (related formats - .lha and .lzs) old PC/Amiga format that is still quite popular in Japan, largely replaced by .arc and .zip elsewhere; decompressed via the Stuffit Engine 4.5 and StuffIt Deluxe 4.5, macunpack, LHA Expander 1.0.3, French KISS 2.2.0 and MacLHA 2.2.1 (which also allows compression).
.pit +
old [~1989] Mac compression format created by PackIt programs, replaced by .sit. In general, a program that handles .sit files can decompress .pit files as well.
.pkg +
AppleLink package format, replaced by .sit. Decompressed by all present Mac StuffIt programs.
.rar
A DOS compression format. Decompressed by MacUnRAR.
.sea +
A special version of a Mac compression format that decompresses itself when opened. The most common .sea files are Stuffit, Compact Pro, and Disk Doubler. On the PC Aladdin Expander will expand Stuffit SEA files.
.shar
Unix shell archive. Decoded by Unshar.
.taz
another name for .tar.Z
.tgz
another name for .tar.z and .tar.gz (do not confuse with .tar.Z).
.txt (.abs, .doc)
ASCII text file. There is a slight differance between ASCII text files of Mac, PCs, and UNIX systems which can cause problems when trying to read them. Mac ASCII uses carrage returns, UNIX uses line feeds, and PC uses both.
.z
Suffix used by both Unix pack and early [~1993] Gzip files.
Due to confusion between these compression methods and Unix 'compress' suffix (.Z) it was abandoned in favor of the .gz suffix. Unix pack itself has been effectively replaced by both Unix compress and Gzip.
.zoo
old [~1989] PC/Amiga format, replaced by .arc. Decompressed by MacZoo and MacBooz.
After decoding and expanding a file I get an unknown document file. How do I open this file?
The best thing to do is to try and see if there is any way to figure out what -broad- type of file it is: word processor, picture, sound, or movie.
Word processor
Tex-Edit Plus <http://members.aol.com/tombb/index.html> will read most of these out there though some will require Adobe Acrobat Reader (.pdf) or a commerical word processor such as MS Word or WordPerfect.
Pictures
GraphicConverter (Shareware, $30-$35, /info-mac/gst/grf/, http://www.lemkesoft.de/) is one of the most powerful shareware graphic programs for the Mac. It is able to open 88 graphic formats, edit them, and save in 35 of these formats including .gif, .tiff, .png <http://www.cdrom.com/pub/png/>, and .jpeg. But even it cannot view propriety formats such as used by Photoshop or Canvas or relatively obcure formats such as .cel or .ecc.
More details on graphic formats in general can be found in the PC Webopaedia <http://www.sandybay.com/pc-web/graphics_file_format.htm>.
Sound files
Sound App 2.5.0 (Freeware, /info-mac/gst/snd/) will play most sound formats out there including .mod, .wav, .au, and .aiff.
Movie files
Varies depending on the movie file type. Quicktime 3.0 is able to view .mov, .mpg (PPC Macs only), .fli/.flc, and .avi (up to IV 3.2) formats.
GraphicConverter is able to view .Ani, .dl, gif, and .fli/.flc formats.
|
