The Freemap blog, revisited

One of the things that isn’t great about OTV at the moment is the rather clunky file upload system, something that one or two people have mentioned on the OSM mailing list too. Would be much nicer to allow multiple file upload, something like this:

http://valums.com/ajax-upload/

So that’s what I’m planning to do next. In conjunction with this, I’ll also be changing the workflow on how to group photos, e.g. link two photos in opposite directions from the same place. Rather than the current approach of doing it at upload-time, the new approach will involve uploading all photos to the server then doing the organisation as a separate step.

Thanks for all the comments so far.

Quick update on OpenTrailView, which I’ve started working on again after a break in May and June due to work commitments. Anyway, gave a presentation on OTV at State of the Map in Girona earlier this month, and got a bit of interest… in particular, one suggestion made by two or three people was to allow users to contribute ordinary photos as well as panoramas, reason being that the process of creating a panorama can be a little long winded.

So now you can do that on the upload interface. You can contribute single georeferenced photos, panoramas or sets of photos taken at the same point. Since one of OTV’s aims is to allow users to “preview” paths or trails before walking them, it would make sense
to allow users to virtually “walk” the trail in both directions. So if contributing ordinary, non-panoramic photos, it makes sense to take *two* photos from the same spot, facing in opposite directions. The upload process allows you to do that, you can upload the first photo in a pair and then add a photo to the same ’set’, specifying its orientation with respect to the first.

Another recent development is that you can now link photos together (select the “Link” option from the main page) to create a route.

For the moment OTV should be very much considered a “work in progress”, in particular I want to do quite a bit of work on the viewing interface which may be prone to interruptions in service. However, the facility to *upload* photos should continue uninterrupted. Check here for updates!

Also updates on my other projects: to be honest, work and other “real life” commitments have been really significant this year, so it’s very unlikely, unless I get a lot of interest, I’ll be working on any other OSM-related projects other than OTV (sorry about that!) However, longer term I’m hoping to tie in the “Footnav” mobile navigation project in with OTV, and if anyone is interested in working some more on the Freemap code, that’s available in OSM SVN.

First of all, OS release the Vector Map District data today, yet to have a look at it but apparently it does feature wood boundaries quite accurately which is good news.

On to the main point of this post though, which is to announce the latest developments on OpenTrailView. There’s now an easier upload system, where you can upload a series of panoramas at once. You can also set the latitude and longitude of each panorama while uploading by clicking on the accompanying map, or, if you also provide a GPX trace, you can automatically calculate the position of your panoramas using the timestamps of the GPX trace and the panoramas themselves.

Main thing though now is the display of the actual panoramas themselves. I’m now using JSPanoViewer, a very interesting, plugin-free pure JavaScript panorama viewer developed by Bart Van Andel. This looks very promising: performance is pretty decent on Firefox though slower on Opera or Chrome. View the panoramas by clicking on the photo icons.

No new developments on Freemap itself just for the moment though am still hoping to switch servers: watch this space!

So the OS releases free data today. I have to admit though that it is a very disappointing set of data, and, to be honest, doesn’t really cover anything that is either in OSM already or could easily be surveyed for OSM anyway. The lack of countryside data such as exact courses of rights of way, and more particularly, field and wood boundaries, will continue to make countryside mapping harder than it would be otherwise.

This is the sort of data OS really need to release. Not A roads, which are on OSM already. Not road names, which can easily be checked up on the ground. But things like field boundaries, which are almost impossible to survey on the ground without incurring the wrath of some irate landowner. Or wood outlines or countryside landuse. These are sort of things we really need.

Oh well, at least I can continue to spend my Sunday afternoons surveying footpaths for OSM, I guess. And it’ll give me more motivation to develop OpenTrailView as a navigation aid which can offer much more than a basic map…

Have mentioned this on the OSM list already but I’d like to announce a new project: OpenTrailView. Having been – I have to admit – just a little impressed by the current state of Google Street View, with the whole of the UK now covered, it’s inspired me (or re-inspired me, I was playing around with this idea about a year ago) to think about developing something similar for the countryside, hence OpenTrailView. (why “Trail”? Well it sounds better than something like OpenPathView and hopefully it’s a bit more internationally-friendly too!)

The idea is to build a collection of geolocated panoramas (i.e all-round photographic views from a particular point) which will, hopefully, be sufficient to produce a StreetView like effect. You take a series of photos in all directions (maybe 8 to 12) from a given point, then use photo-stitching software such as Hugin to stitch them together. Anyhow, more on the OpenTrailView site. At the moment you can contribute a panorama and then use an OSM map to align with a path.

One last point if you’re interested in contributing to any Freemap projects. I’ve now added the source code for all the main things I’m hosting on the Freemap server to OSM SVN, under sites/free-map.org.uk, so if you’re interested in contributing, look there!

Am making progress towards the new version of Freemap, which for the first time will have a version number – 0.5 (why 0.5? Basically it’s the fifth major revision, starting with the pre-OSM days back in 2004, more on that when it launches) and hopefully, using a combination of home rendering and the SUCS tile server, I will manage to cover the whole country.
From a developer perspective I also now plan to use the main OSM SVN repository to hopefully invite contributions.

What I’d like to get though from you is a few feature requests. I’d like to make Freemap a genuinely useful site, basically “the OpenStreetMap site for walkers in the UK”, in the same way that the cyclemap and cyclestreets serve that purpose for cyclists. So if you do have any features that you think would be really good for Freemap to have, please let me know either on this blog or on nick dot whitelegg at yahoo dot co dot uk.

Note that due to server constraints it may not be possible to deliver features which require a full
installation of the OSM database on the server. However, if your suggestions do involve that and they’re interesting, do let me know anyway as I may be able to implement them in a small test area.

Have now finished with my initial experiments on home tile creation and can now give you an idea of what will be on the new Freemap. I’ve come up with a style I like which combines the existing Freemap style for rights of way and contours with the current full OSM style – so for the first time, Freemap will show the full range of landuse shading shown in the main OSM tiles.

Bad news perhaps is the area of extent. It takes five hours on this quite powerful machine to generate the tiles in the bounding box 2W, 50N, 0, 52N for zoom levels 10 to 15, and then almost an hour to upload the 500MB odd of data to the Freemap server. This basically equates to an area which just about includes Swindon, Oxford, London, Brighton, Bournemouth. This is therefore the absolute limit of data I can deal with. However, what I’ll probably do is ditch zoom level 15 (as that takes the majority of tile generation time and file upload size) and expand the area of coverage a little, west to 2.5W to cover a bit more of Wessex and the Cotswolds. As indicated earlier I may well also add in the Lake District and Snowdonia. Sorry guys if you’re out of this area but with the limited processing power available to me, that’s all I can do! However, as said earlier I hope to cover the rest of the country via the Swansea University Computing Society webserver – though as I don’t have admin rights it may be a little more restricted.

And if you do have the technical know-how (and if you follow the wiki, it’s really not hard to do your own tile generation, though it does require familiarity with a command-line) I’d urge you to set up your own Freemap server for your own part of the country. This seems to me the most effective way to deal with resource issues: distribute the load to several machines, each covering a small part of the country but when combined together, able to produce full maps of the whole UK. Do get in touch if you’d like more help on this!

Another update from last night. Have got contours working! The standard contour-importing procedure into PostGIS was completely painless and now I’ve got some nice tiles being rendered. Just want to tweak the Mapnik rules and it should be ready!

One issue though is the large amount of data I have to upload onto the Freemap server. Luckily my broadband usage is unmetered after 6pm (soon to be 7pm while we get a useless free hour between 6am and 7am when we’re all still in bed … nice one, BT!) so that’s not too much of a problem – but the bzipped tar file of a significant area of S England between zoom levels 10 and 16 still came to 250MB odd. So, I’m sorry to say, I am likely to have to restrict rendering to certain areas, probably at this time southern England plus one or two other interesting areas like Snowdonia or the Lake District. Nonetheless the rest of the country will hopefully continue to be served, either by the SUCS site or by the old tiles on Freemap. Sorry if this isn’t absolutely ideal but it’s an unfortunate side effect of this being a hobbyist site and I think it’s more important to do a small area well rather than a large area badly. Anyway, a) it’s not that hard to set up your own server for your own area (honest!) and b) who knows, if I develop Freemap for S England it might attract sources of income to expand the area covered so be patient!!!

More soon but look out for a relaunch and new style tiles in the near future!

These memory issues on Freemap have really been bothering me, so spent much of the day trying to find a way round it. First of all I used Osmosis to try and reduce the amount of data in the .osm file to what was absolutely necessary for Freemap. I did this and successfully imported a recent (March 1st) planet extract to the Freemap database, however the file size was not that much reduced and therefore this is unlikely to provide a long-term solution.

So I finally took the plunge and decided to install a full Mapnik setup on my home machine – PostGIS and all – with the intent of rendering the tiles at home, where I have the memory and processing power, and upload them to the server, where I don’t. This actually wasn’t quite as painful as I remember from last time (when I set it up on Freemap more or less three years ago to the day) – I encountered one or two problems (setup: Ubuntu 9.10, Postgres 8.3, Mapnik 0.6.1, and latest SVN version of osm2pgsql) but both were resolved by looking at the wiki. Two problems might not be immediately apparent on this setup (they caused me difficulty at first) so I’d like to draw attention to them: section 5.1.7 (Authentication failed) on the Mapnik OSM wiki page, and the need to run the 900913.sql file included in osm2pgsql (see the osm2pgsql README).

So have successfully rendered some tiles using the default Mapnik style. Next task will be to tweak them for the Freemap style and add in contours a la CycleMap. One problem of course will be the size of the archive I have to upload to the server, but for some of the UK at least, with luck the memory problems will be solved! In terms of the database on Freemap, I’ll probably keep it for selected, relevant POIs only (to allow the maintenance of the search facility) but the ways will be dropped.

So overall a good day! Only thing was, it was 3pm by the time I finished that, so while I did manage to get out for about three hours, I couldn’t get to a place where I could do some new mapping! Still, only three weeks till BST and we can finally lay the winter to rest…

A bit of an update on the Footnav project, which aims to be a 3D mobile navigation tool for countryside users. In the past few months Android has become a serious contender in the mobile world, and being the most viable mobile operating system if you don’t want someone restricting redistribution of your app, has become my main focus of interest in terms of mobile development. Got myself an Android phone recently too so fancy having a play!

Therefore I’m shifting Footnav development from C++/Qt to Android. Haven’t done any Android development before, though, so as a first step I’m aiming to produce a simple Android app called “Fix My Paths!” The aim of this will be to allow users to report footpath problems using their phone and either upload them direct to the Freemap server in the field, or save on the SD card for upload at home. Then I’ll probably port Freemap Mobile to Android, before converting the WebGL 3D code (something that should now be a lot easier now I’ve had a taste of the sometimes-esoteric ways of OpenGL ES…)

Would also like to keep the WebGL 3D viewer going, though as always my time is limited and it’s a case of which project is interesting me the most at the moment….