Empty WAY and NODE tables

Mar 23, 2012 at 6:34 PM

I tried to import the Planet file (Planet-120314) and it doesn't seem to have worked.  I got data in the tNodeTag, tWayCreation and tWayTag tables, but tNode and tWay are empty. 

Also, in the Import steps, after Create the Database Structure, the step 'Read all node from the OSM to the database' completes in 2 or 3 seconds.  That can't be right!

The logs don't show any obvious error messages.  Any ideas?

Coordinator
Mar 23, 2012 at 7:18 PM
Edited Mar 25, 2012 at 8:09 PM

Hi!
Installed today the Microsoft SQL Server 2008, will have a look at it Sounds very strange…

Greets
christian

Mar 23, 2012 at 7:26 PM

I should mention that I used SQL Server 2012 Standard Edition RTM (MSDN Version).  Thanks!

Coordinator
Mar 23, 2012 at 7:33 PM
Edited Mar 25, 2012 at 8:10 PM

Ok thats now really strange , thought It was a SQL Server 2008 Problem !
But I remember that I changed something in the code on Thursday,
because I had a mistake in the code.

Maybe you update your version to the newest revision ?

Mar 23, 2012 at 8:35 PM

OK, I've tried it with three versions and I'm getting the same results.  Originally I was using 0.2.0.  I just downloaded and tried both 18843 and 18844 and still get the same result.  Here is the log file:

3/23/2012 4:27:11 PM - -----------------------------------------
3/23/2012 4:27:11 PM -
3/23/2012 4:27:11 PM -
3/23/2012 4:27:11 PM - - Logging started . . .
3/23/2012 4:27:11 PM -
3/23/2012 4:27:11 PM -
3/23/2012 4:27:11 PM - -----------------------------------------
3/23/2012 4:27:25 PM - Starting Task: Creates the database structure
3/23/2012 4:27:27 PM - Finished Task: Creates the database structure
3/23/2012 4:27:27 PM - Starting Task: Read all node from OSM to the database
3/23/2012 4:27:27 PM - Starting Task: Read all ways from OSM to the database
3/23/2012 4:27:27 PM - Task is waiting until all running tasks finish...
3/23/2012 4:27:27 PM - All tasks are finished
3/23/2012 4:27:27 PM - Creating indices on tNode...
3/23/2012 4:27:27 PM - Finished Task: Read all node from OSM to the database

I stopped it once it skipped the Node import.

Could it be checking for required disk space?  I have 380GB free, and I assume the database will be smaller than the source XML (which is 308GB), but maybe I'm wrong about that.

Mar 23, 2012 at 8:58 PM

I've debugged it to find out what's happening, and it looks like a data format issue.  When the ReadNodes function starts to read the file, it does this:

reader.MoveToContent();
reader.ReadStartElement("osm");

while (reader.IsStartElement() && reader.Name == "node")

but reader.Name is 'bound' instead of 'node'.  This causes the reader to break the while loop.

Does the project not support reading the full Planet file, or is there something I need to do differently?  Thanks!

Coordinator
Mar 25, 2012 at 8:08 PM
Edited Mar 25, 2012 at 8:10 PM

Hi!

Please try the latest version, should work now…
It was a bug which I stumbled over myself last week, if a file contains <bounds> the importer didn´t work right,
now it should !
(GeoFabrik.de – Europe.osm didn´t contain this node – which I already imported!)

Would be nice to hear if it works now.
(Hope the version works at all, currently i´m only able to code from Wednesday until Friday
and there is not much time…)

Greets Christian

Mar 26, 2012 at 1:25 PM

Thank you for all your hard work!  I downloaded this new version and it is running now.  Both the Nodes and Ways are importing.  It will take all day to import the whole file (I've got it running on an old PC for testing) so I'll let you know when it completes.

Thanks again!

Developer
Apr 26, 2012 at 11:13 PM

I ran into the same issue as well trying to load california.osm.pbf with 0.2.0.  I used source version 19615 and then things loaded.   I am now looking for the type number mapping, so I can try to interpret the tags.  Thank you for writing this piece of code. 

Coordinator
Apr 29, 2012 at 7:29 AM
nabramovitz wrote:

I ran into the same issue as well trying to load california.osm.pbf with 0.2.0.  I used source version 19615 and then things loaded.   I am now looking for the type number mapping, so I can try to interpret the tags.  Thank you for writing this piece of code. 

Nice to read, that this piece of code helps somebody.
Would be nice to read from you what you are doing with it!...  =) 

Developer
Apr 30, 2012 at 4:02 PM

We are considering using openstreet as a piece of the search criteria for finding real estate.   For example, I want to be near a train station or I want a golf course to be near by or I want to be away from highway and train noise.   Your code allowed us load up the openstreet data and start evaluating it right away since we are using MS sql server as our database.

Coordinator
Apr 30, 2012 at 8:32 PM
nabramovitz wrote:

We are considering using openstreet as a piece of the search criteria for finding real estate.   For example, I want to be near a train station or I want a golf course to be near by or I want to be away from highway and train noise.   Your code allowed us load up the openstreet data and start evaluating it right away since we are using MS sql server as our database.

Really cool!

It would be nice if you can contribute your improvements to the community or link to the projectwebsite if you are using it or going to use it. :-)