Extracting Data From The Overpass API Extracting Data From The Overpass API

December 23, 2021

laravel openstreetmap

I extracted an XML dump from the overpass-api.

My plan is to import this data to a database so I wrote a php script to convert this data from XML to Json and then output the data in the terminal.

<?php

$xmlDataString = file_get_contents('osmData.xml');
$xmlObject = simplexml_load_string($xmlDataString);

$json = json_encode($xmlObject);
$phpDataArray = json_decode($json, false ); 

$counter =0;

foreach($phpDataArray as $value){
    foreach($value as $row){
        $name = "";
        $twitter = "";

        foreach($row->{'tag'} as $attributes){
            if($attributes->{'@attributes'}->{'k'} == 'name'){
                $name = ($attributes->{'@attributes'}->{'v'});
            }

            if($attributes->{'@attributes'}->{'k'} == 'contact:twitter'){
                $twitter = ($attributes->{'@attributes'}->{'v'});
            }

            if($attributes->{'@attributes'}->{'k'} == 'website'){
                echo  ($row->{'@attributes'}->id );
                echo " ";
                echo  ($row->{'@attributes'}->lat );
                echo " ";        
                echo  ($row->{'@attributes'}->lon );
                echo " ";
                echo ("\e[1;32;40m" . $attributes->{'@attributes'}->{'v'} ."\e[0m");
                echo " ";
                echo "\e[1;33;40m" . $name ."\e[0m";
                echo " ";
                echo "\e[0;34;40m" . $twitter ."\e[0m";

                echo PHP_EOL;
                $counter++;
            }
        }
    }
}

echo "total records with website: " . $counter . PHP_EOL;

Here is a sample of the output which shows entries that have a website url.

The green values are the website values, the yellow the names and the blue ones the twitter fields. Some of the websites arnt websites, many dont have the http/https prefix. Some of the entries dont have names. The twitter links arnt in any standard format, full url, @twitterhandle or just the twitter handle.

I will develop this script further to extract more info from the xml and work out how and what to store in the database. Stay tuned.


If you would like to contact me you can either use this form on londinium.com or via Twitter @andylondon