gen2shp - converts ArcInfo generate format to shapefile format

This is the new homepage of gen2shp (see the previous homepage to get sources and DOS-binaries of version 0.2.3).

What is it?

gen2shp is a simple C-Progam which can read the format required by the ArcInfo generate command. The data in this file will be converted to the shapefile format which actually means to three files: .shp, .shx and .dbf.

Why was it developed?

Within the framework of my PhD thesis I have implemented an automatic processing scheme to create ready-to-use data sets for GREAT-ER (Geography-referenced Regional Exposure Assessment Tool for European Rivers). Among these are several geodata sets for use in Shapefile format. As a starting point all data are requested in a specific text file format. For the further processing of these file and (as a part of it) for the creation of geodata in Shapefile format to be loaded in Thuban I had three requirements: All parts of the methodology have

to be independent of any proprietary product. Hence, the whole methodology can be made available as Free Software.
to be platform independent. Hence, almost anyone who is interested can execute the methodology.
to have a commandline interface. Hence the command sequence can be handed over to a makefile and this way be made 100% automatic.

In case you need a gen2shp solution based on Avenue, I recommend to use the Free ArcView scripts by Ron Wardenier. These are licensed with the LGPL.

History

Changes since 0.3.1 (only in CVS)

* added example to usage (thanks to Tom Callsen for this hint)
* fixed problem for empty polygon input files (bug found by Michal Lewandowski)
* Rearrangement of files for CVS, some code cleanup.
* Makefile: Fixed a problem in the sequence of parameters for CC (fixed by Jørgen Binningsbø).

Changes from 0.3.0 to 0.3.1

Just a better packaging, no changes in the source code.

Changes from 0.2.3 to 0.3.0

Added multi-part support for polygons (trying to handle islands within a polygon correctly). This new feature has passed some of my tests, but it should not be trusted blindly.

This version compiles differently, you should have dynamic version of shapelib or modify Makefile to compile in old fashion of V0.2.3. Will provide DOS-binaries if I find some time.

Thanks go to Ron Wardenier and Branko Kosuta who are responsible for pushing me to implement the new feature.

Changes from 0.2.2 to 0.2.3

ArcInfo uses "D" instead of "E" for the exponent representation. This was not considered in the previous version, it was necessary to replace "D" by "E" before issuing gen2shp. This is fixed now.

Previous releases only accepted "," as delimiter. Now also spaces are accepted. Furthermore any sequence of delimiters is treated as one delimiter (this was different in the previous version!).

Thanks to Markus Neteler, University of Hannover and to a gen2shp user named Eric.

Changes from 0.2.1 to 0.2.2

ArcInfo handles the keyword "end" case-insensitiv. Now gen2shp does, too. Thanks to Doug Nebert of the Federal Geographic Data Committee for that hint.

Changes from 0.2.0 to 0.2.1

Just a typo and test with new revision of shapelib (1.2.4 to 1.2.5). Furthermore, DOS-executables are now included.

Current version

The current version is 0.3.1 and can handle points, lines and polygons. It is successfully tested on a Solaris, Linux and DOS, but there is no reason why it should not work on any other.

gen2shp does the job properly I needed it for. If you have any problems applying gen2shp send me an email. By one of us or jointly the problem will surely be solved.

You will need the shapelib version 1.2.8 (or higher) which is freely available (see requirements and installation below).

Future Versions

Many features are conceivable:

Consideration of 3D-Objects
Development of 'gen2shp's counterpart: shp2gen
... (tell me)

Usage

usage: gen2shp outfile type < infile
        reads stdin and creates outfile.shp, outfile.shx and outfile.dbf
        type must be one of these: points lines polygons
        infile must be in 'generate' format

Copyright and License

See head of file gen2shp.c for details.

I have used a module (utils.c) being a part of txt2dbf 1.0.2 ( dBase Tools). The author is Frank Koormann. He made the source freely available in the common sense - thanks to Frank.

Requirements

shapelib 1.2.8

Developed by Frank Warmerdam. Download from Shapelib Homepage.

Licensed as Free Software. Thanks to Frank.

Download

gen2shp-0.3.1.tar.gz (16 KB) or gen2shp-0.3.1.zip (17 KB)

Binaries for Windows prepared by Branko Kosuta <bkosuta@hotmail.com>: gen2shp-0.3.1.bin.msw.zip (41 KB)

Binaries and sources for an enhanced Windows-Version by Rusman Rusyadi <rusmanr@yahoo.com>: Gen2Shp-0.3.1-win32.zip (33 KB). This was compiles with borland c++ builder and has been tested on W2000.

Thanks to Branko and Rusman.

CVS

Online:	[ gen2shp source code via viewcvs ]
Read access:	CVSROOT=`:pserver:anonymous@intevation.de:/home/freegis/jail/freegisrepository` Password: `anonymous` ; Module: `gen2shp`

Installation steps

For all standard operating system, the following steps should work:

Download shapelib 1.2.8 (or higher) , install and compile it somewhere on your system - gen2shp prefers the dynamic library (manual work needed for static binding, not much work though).
Download gen2shp, unpack files somewhere on your system.
Execute 'make'. Everything should work.
Execute 'test' to check whether something wicked happens.

generate Format

You may refer to ArcInfo documentation (if available). For Polygons, a special syntax allows multi-part description: omitting an ID after an END, a new part of the previously started polygon is created rather than a new polygon.

Examples should illustrate format as well:

Example: points

8,-1.405877,53.721508
13,-2.045479,53.731064
14,-1.984840,53.731972
15,-1.925755,53.710373
16,-1.889399,53.706749
17,-1.859130,53.696831
18,-1.730477,53.681328
19,-1.719866,53.683102
20,-1.604871,53.673790
21,-1.547393,53.669090
end

Example: lines

12033
-1.352874,53.629650
-1.352463,53.629875
-1.351808,53.630165
-1.351263,53.630173
-1.350566,53.630314
-1.349868,53.630444
-1.349170,53.630573
-1.348428,53.630669
END
26456
-1.360715,53.624237
-1.358755,53.623821
-1.356220,53.623493
-1.352882,53.623207
-1.350488,53.623516
-1.349195,53.624039
-1.347095,53.625870
-1.349000,53.628876
-1.348428,53.630669
end
9033
-1.371616,53.643658
-1.367998,53.642830
end
9037
-1.395607,53.669220
-1.394491,53.669979
-1.393896,53.671398
-1.393909,53.673672
-1.390823,53.674545
-1.383970,53.675240
-1.381641,53.674030
-1.375354,53.673252
-1.371793,53.672478
end
end

Example: (simple) polygons

1
0,0
1,0
1,1
0,1
0,0
end
2
10,10
10,11
11,11
11,10
10,10
end
end

Example: multi-part polygons (with islands)

701
3,9
4,5
8,4
9,7
6,9
3,9
END
7,7
4,7
6,6
7,7
END
702
13,9
14,5
18,4
19,7
16,9
13,9
END
17,7
15,7
15,5
17,5
17,7
END
23,9
24,5
28,4
29,7
26,9
23,9
END
END

Frequently Asked Questions (FAQ)

1. Howto create attribute (dbf-) files for shapefiles?

gen2shp can not process other data than the IDs and coordinate information of geographic objects.

This example illustrates principle ways how to create the .dbf-file that contains the attributes for a shapefile.

Consider you have point information with some attributes in one file:

bash-2.05b$ cat data.txt 
# ID,X,Y,Value1,Name,Value2
8,-1.405877,53.721508,42,A Name,6.8
13,-2.045479,53.731064,12,Yet another Name,2.7
14,-1.984840,53.731972,1234,No Name,3.14

First step is to separate the coordinate data from the attribute data (note, that the grep command eliminates lines with a '#'):

bash-2.05b$ cut -d , -f 1,2,3 < data.txt | grep -v '#' > data.gen
bash-2.05b$ cat data.gen 
8,-1.405877,53.721508
13,-2.045479,53.731064
14,-1.984840,53.731972

bash-2.05b$ cut -d , -f 1,4,5,6 < data.txt  > data.att
bash-2.05b$ cat data.att
# ID,Value1,Name,Value2
8,42,A Name,6.8
13,12,Yet another Name,2.7
14,1234,No Name,3.14

The shapefile is created now with gen2shp:

bash-2.05b$ gen2shp test points < data.gen 
bash-2.05b$ ls -l test.*
-rw-rw-r--    1 jan      intevati      113 15. Nov 18:30 test.dbf
-rw-rw-r--    1 jan      intevati      212 15. Nov 18:30 test.shp
-rw-rw-r--    1 jan      intevati      132 15. Nov 18:30 test.shx

The .dbf-file needs now to be replaced by a one that contains the attributes: First, change the name of the ID column to the column name used for the shapefile. You can check this with dbfdump (part of the shapelib package):

bash-2.05b$ dbfdump test.dbf | head -1
    test-id

bash-2.05b$ cat data.att
#test-id,Value1,Name,Value2
8,42,A Name,6.8
13,12,Yet another Name,2.7
14,1234,No Name,3.14

Now create the dbf-file with txt2dbf:

bash-2.05b$ txt2dbf -v -I4 -I5 -C20 -R5.2 -d ,  data.att test.dbf
txt2dbf 1.0.2, 09.09.1998 by Frank Koormann
 data.att --> test.dbf
 FIELDS:
 NAME		TYPE	LENGTH	DECIMALS
 test-id    	N	4	0
 Value1      	N	5	0
 Name        	C	20	0
 Value2      	N	5	2
 RECORDS:
 [1] [2] [3]
 3 Records written into test.dbf

There are many, many other ways to establish what is demonstrated here.

2004/12/20