PlanetEye

Travel Beyond Words

Archive for the ‘GPS/Geotagging’ Category

Thursday, December 3rd, 2009

A Realtime Twitter Map

Juan Gonzalez
Vice President, Technology

When we heard that Twitter was launching a geolocation API, we figured given the volume of people using Twitter every day, the total number of geolocated tweets would soon become unmanageable in a map interface. Precisely the kind of challenge that we like to take. I’ll share how we solved the problem and why this is important to us, but first take a look at our Live Twitter Map (hint: zoom out/in or move and you should see new markers appear every now and then)

Live Twitter Map

Live Twitter Map

How to build a Live Twitter Map

While we don’t have a partnership with Twitter, we were able to use their Streaming API to get a good sample of tweets in realtime. They must have a way to pull all geolocated tweets as Bing Maps started using such a feed this week. Regardless, with our approach we are getting up to 400 Kbps of data consistently and we have to filter such stream to find tweets that are geolocated.

Once we have the geolocated tweets, we can run them through our clustering algorithm and store them temporarily (we keep up to seven days of data). As you can see, our approach to visualize large volumes of data on the map scales better than trying to put a marker for each data point. Also worth mentioning is the fact that the clusters refresh in realtime as new data is added to the system. Ironically, while testing the implementation of Twitter on Bing Maps I was consistently getting delays of up to 5 minutes for a given tweet to appear on the map, while it would show in our map within seconds.

Another advantage of our approach is that in any given region of the map our clusters will tell you how much data there is in total. It seems that other maps are cursed by the limitations of having too many markers on the map and they usually opt for displaying “the 50 most recent”. Certainly nobody will want to read every single data point (especially if they are tweets), but being able to see the areas where the majority of the tweets are generated is in itself useful information.

Why build a Live Twitter Map

Over the last couple of years we’ve learned to integrate all kinds of data into our system for the benefit of our clients. When we consider all hotels, restaurants, attractions, activities worldwide, we end up with a very large data set that needs to be managed efficiently if we are to make any meaningful recommendations. In every single instance the process to acquire the data consisted in negotiating some sort of XML feed or parsing a delimited file, which would always create a delay between when the data was updated on the source and when it would be available on our maps for consumption.

Now we know that it is possible to apply the same principles that we used to create a Live Twitter Map to continuously update data from our various partners and make it available to our subscribers. Imagine being able to see on a map when tickets to a given event become available so you can be the first in line to grab them. Or noticing a new promotion that would give you a great discount on a hotel you are planning to book. With our technology to acquire and process geolocated data along with our mapping techniques we have a killer solution for clients in the travel space and we are working hard to bring these solutions to the end users.

If you have an interesting data set that needs to be put on a map to enable easier discovery, we want to hear from you.

UPDATE: On December 9th, Twitter’s Director of Platform Ryan Sarver announced that the firehose will become available to everyone. In principle this would allow applications like ours to become aware of every single tweet and use that information to build custom applications around them. In our case, the possibility of processing every single geolocated tweet would allow us to showcase the power of our mapping platform with the largest test case there is.

Monday, November 30th, 2009

PlanetEye Traveler: our online travel magazine

Juan Gonzalez
Vice President, Technology

On the wire today:

PlanetEye is pleased to announce the launch of PlanetEye Traveler, a feature-rich online travel magazine that is packed with unique and advanced user-friendly functionality. Designed for consumers and travel industry professionals, this award-winning portal features eye-grabbing editorial and rich visual content which is provided by dozens of travel writers from around the world.

You can read the full press release, but I thought it would be interesting to recap how we got here:

Oct 2007: as par of our original launch, we invited a select number of local experts to write for PlanetEye covering their cities. At the time we knew their insight would help us provide much better recommendations. Because we wanted their writing to be cross-referenced against our city guides, we used our own tools that allowed them to map whatever they were writing about.

Oct 2008: after a year of great writing we realized that in order to scale this operation we had to embrace open standards and we took the first step by endorsing microformats. Our thinking at the time was that if we had a neutral approach to recognize location information in blog posts, we could extend our program to other bloggers beyond our own network. It didn’t take long before our own bloggers realized that the tools we created for others were in fact much easier to use than our proprietary system. And so the journey began.

Jan 2009: after rewriting our entire toolset we released all our archives into blogs running on Wordpress. Since then all our writers have been using nothing else but Wordpress with a custom plugin that does all the magic: every single time a writer tags a post with a location through our plugin, we embed a Google Map on the post making them more useful to the readers, but we also associate that post with our destination database so we are able to do a few more interesting things behind the scenes.

Oct 2009: With the launch of PlanetEye Traveler, not only we bring our entire team of writers under one site to create a very unique online travel magazine, but we also showcase what readers can expect in the future from ours and other similar magazines: travel articles used to be completed detached from context. With our technology it is now possible to bring relevant location information (such as the map) into the article and vice-versa. Really impressed with that Restaurant review in your city? You may soon be able to check the menu, read other reviews, see the latest buzz or even reserve a table.

Of course, the inspiring travel writing will continue to be the most important aspect of our travel magazine. Whether you are catching up on the latest events around a city or the highlights of the season at your next destination or finding incredible travel stories from abroad, we think we have a great source for you.

Friday, November 13th, 2009

Geocoding with GeoAPI

Juan Gonzalez
Vice President, Technology

Trying to keep up with the latest developments in the Geolocation space is though, specially when you get all those tweets from friends checking into a new bar every few minutes with Foursquare. Not surprisingly the integration of some forms of GPS into mobile phones has made us into full “blue-dot” converts. As with most technologies, the basic test case has to be fundamentally simple for the masses to adopt it and in the case of geolocation that test case is “Here I am”.

The geolocation game is a lot harder than it seems and this has nothing to do with displaying a dot on a map. Just ask any geocaching fans and you’ll understand how the map and the GPS are the easy part of the equation. Finding your way through terrain, cities, dealing with unexpected (unmapped) characteristics of the terrain is what makes it interesting and fun. For us dealing with geodata day after day, the challenges start with ethereal questions such as “who owns a coordinate” and branch into a number of data crunching issues.

One particularly interesting challenge that will likely continue to draw the attention of many players in the space is how to geocode information that wasn’t properly tracked. Our ability to do so will reveal a new way to correlate such information on a new geographical index, bringing another relevancy metric into search. Being able to do a reverse geocode, specially with a very robust database of place names is a great starting point. Already Yahoo had put together a very valuable set of tools to this effect. Now with the launch of GeoAPI from Mixer Labs, the game gets more interesting.

Wednesday, November 5th, 2008

Crowdsourcing GeoData

Juan Gonzalez
Vice President, Technology

Last week, Flickr announced the availability of shape data through their API. I had the opportunity to share the Where 2.0 podium with Dan & Aaron last May as they presented their progress in leveraging the millions of geotagged photos on Flickr.

The Significance of Geographical Information in Photos was an important milestone in explaining how meaningful geographical information was possible without using the services of sophisticated geo-providers. With the recent announcement, they’ve delivered a service that provides an alternative to expensive geographical boundary information.

Why is this important?

Building a service around geocoded data has become easier with the proliferation of mapping technology…..as long as all we do is display data on a map. If the service tries to do any calculations based on the geographical context, the maps are useless as they are nothing else but an image.

To really leverage the hundreds of thousands data points in our database to deliver great travel recommendations, we need to be able to answer questions such as “Is this hotel located in the city of Toronto?” or “What is the closest town with a good variety of restaurants?” or “Can I get a similarly-priced hotel closer to downtown?“.

Aside from some serious technology to manage large volumes of data, the key to solve these is what is known in the GIS world as shapefiles. While there are many sources for them, very few are available at low cost. Until now.

At PlanetEye, we’ve experimented with similar methods to the ones described by Flickr, and have seen very interesting results. For example, the following map shows the aggregate of all interactions with our map around the city of Toronto.

While we don’t have boundaries built into our application, from this aggregate it is possible to visualize approximately where the city borders with others around. And every time we add a new data point to our database, the picture gets better.

The only downside of this approach is that the boundary information is not accurate. Some cities may appear to be larger than they are only because we don’t have enough information on other cities in the vicinity. But it is safe to assume that as more people use our application, these deficiencies will be corrected.

If you’re interested in the future of the GeoWeb and have something to contribute, remember we have our GeoSocial meet-up tomorrow (November 6) at 6:30pm @ The Charlotte Room

Thursday, October 30th, 2008

Better Blogs with Microformats

Juan Gonzalez
Vice President, Technology

Knowing that original and recent reviews written by local bloggers would help us establish relevancy on the places for each one of our featured cities, we invited around 50 professional travel writers to produce content for PlanetEye. To keep things nicely integrated into the rest of our site we gave these Local Experts tools not only to write but associate their posts to specific places on the map. This helps the reader understand context and allows us to create a relationship to the article from the place referenced. This has worked really well so far: our collective pool of Local Experts have written over 3,500 articles, 50% of which are associated with a place.

So for example while a review with proper references to location would look like this:

The featured place, in this case a restaurant, would have the proper reference to the original post:

We realize our Local Experts are not the only people with a professional opinion on destinations around the world. We’ve come across hundreds of bloggers that feature terrific content. They all have unique methodologies to scout their cities looking for great destinations, they all are diligent about documenting location and other important information as well as writing a great review. Mapplr.com comes to mind as a good example.

We would love to integrate the wisdom of these reviews in the same way we have done so for our Local Experts: by creating a relationship between their review and the places they are talking about in such a way that the aggregation of this information can create a more interesting profile of a city.

To make this happen, we are endorsing Microformats and have put together a white paper that explains our suggested approach to implement them when writing hotel or restaurant reviews. It contains very simple instructions to get started producing structured content.

Already this month has been an interesting one for Microformats with the launch of Oomph, Microsoft’s Microformats Toolkit and enhanced Firefox support.

Download the white paper.

Dowload the White Paper

Monday, October 27th, 2008

Google Earth on the iPhone

Juan Gonzalez
Vice President, Technology

Google Earth is now available on the iPhone. You can watch a short video demonstrating the main features at the Google Lat Long Blog and read some first impressions by other reviewers such as ReadWriteWeb.

While the level of interactivity is very much as sophisticated as it is on the desktop, even enhanced by the iPhone’s multi-touch interface, long time users of the original Google Earth will miss the features that have made this application a favourite of the geo-enthusiasts: using your own data.

The iPhone version of the application includes the Wikipedia and Panoramio layers, but doesn’t support any means to visualize data sets defined in other KMLs. Surely this feature would make this application very valuable for companies with large and complex sets of GeoData that needs to be taken to the streets.

It is obvious that deploying advanced mapping applications will soon stop being an issue about maps and will become a much more challenging visualization issue. With reduced screen size, the mechanisms to ensure users gain access to large volumes of data with ease, become an important problem.

No wonder Google opted to enable only a couple of layers even though their desktop application has a large number of unique layers available. But what to do when a screen of 480×320 must display a map that contains hundreds, if not thousands of data points? At PlanetEye these are the challenges that we believe deserve attention.

Technorati Tags:

Tuesday, October 21st, 2008

FireFox 3.1 – Geode

Alan
Swiss-Army Developer

Firefox 3.1 is around the corner, and with it comes some features that are particularly interesting to PlanetEye.

Geode is Firefox’s implementation of the W3C Geolocation Specification. Specifically, this allows the browser to let a Web site to know its current location. This opens all kinds of possibilities as we can provide location based services to regular browsers without the user having to enter their location.

Currently, Firefox 3.1 beta provides hooks for inputting geo coordinates. Plugins can be created to get this data from a USB GPS receiver, the GPS chip in a phone, or even Skyhook. Of course, desktop users can also easily set their location manually.

Given that Firefox Mobile will likely be on devices in 2009, this lets developers provide the same level of location services to users on a mobile browser as the browser can now relay the user’s location. Before, developers needed to write a standalone application to take advantage of a phone’s GPS features, which is a daunting task given the number of mobile platforms.

With the phone’s Web browser able to provide geo-coordinates, companies will have the option to write a single Web-based mobile app instead of separate standalone ones for each mobile platform.

This shift won’t happen overnight, but much like the transition that happened with desktop to Web-based applications, one of the last barriers to geo apps on mobile has been lifted. Mobile is still a rapidly changing landscape but the future is looking bright for location based apps.

Thursday, October 16th, 2008

OpenStreetMaps in Toronto

Juan Gonzalez
Vice President, Technology

OpenStreetMaps is inviting the Toronto community to participate in their Mapping party, coming up on October 26, 2008. What is a mapping party?

Mapping parties are events where anyone can come along and join in improving the free world map of everything. We show you how to collect data (survey) and get the data in to OpenStreetMap.

While progress in mapping Toronto and surrounding areas has not been consistent, it seems more people are jumping in to help. The results are already quite important:

Toronto OpenStreetMap

Toronto OpenStreetMap

While we’ve always been mapping enthusiasts, the relevance of this type of events is not only that it demonstrates the effective use of user generated content to provide free alternatives to otherwise costly licensed maps, but that the maps that are generated by OpenStreetMaps enthusiasts are typically better focused on the less travelled roads, trails, cycling routes and points of interest that would be otherwise ignored.

Come join the party.

Tuesday, October 14th, 2008

Announcing GeoSocial

Juan Gonzalez
Vice President, Technology

Back in May, during the Where 2.0 conference in San Francisco, I was pleasantly surprised about how many of the participants were representing companies in Canada.

There was an immediate bond among these people, and conversations about the need to bring these events closer to home. Certainly, the quality and influence of our collective projects seemed to be mature enough. So, with that idea in mind, Mark Evans (one of the organizers of the mesh conference) and I have decided to organize GeoSocial.

What is GeoSocial? Is a group for people interested in exploring the uses of geodata to enhance the relevancy of information on the web and create new means of social interaction.

Brady Forrest and Andrew Turner recently published their first “Where 2.0: The State of the Geospatial Web“, which may help you understand the progress that has been made in what they’ve called the GeoWeb:

A number of trends are driving towards a new paradigm we’re calling Where 2.0:

  • Web 2.0 with its wikis and blogs, is empowering millions to publish and contribute content to open services;
  • More geodata is easily available and freely disseminated; and
  • Mobile devices are becoming location-aware. They are increasingly equipped with wireless internet connectivity and GPS. We can connect from almost anywhere.

We firmly believe the GeoWeb has succeeded in providing an easily understood framework for gathering, analysing and displaying information. The challenges that remain ahead are now about leveraging this framework to create better ways for people to explore and discover their world.

GeoSocial wants to provide an open forum for all people developing ideas, technology and products that leverage the GeoWeb to create advanced social applications that can deliver personalized, geographically relevant content. We also want to invite people from other areas (bloggers?) to participate and discover how these technologies can fundamentally impact their own products.

To get things started we’ll be hosting a GeoSocial meetup in Toronto on November 6 starting at 6:30pm at The Charlotte Room. Or if you’re not in the area look for us on Facebook.

Wednesday, October 8th, 2008

Relevant GeoData

Juan Gonzalez
Vice President, Technology

Geode
By now, people know that mobile phones can figure out your location. Most people also know that some of those mobile phones can do it faster than a GPS or even without a GPS.

Developers have embraced a variety of systems intended to acquire, centralize, broker, protect, use and reuse such location information but most users seem to be content with being a blue dot on a map. Even users without a mobile are going to be equipped with the tools to leverage their location information.

So what does PlanetEye make of the recent launch of Geode by Mozilla Labs?

It confirms that we are developing the right tools for how users will navigate the Web tomorrow. Regardless of which product ends up managing location on behalf of users, it is safe to assume that everyone will soon be capable of sharing this information.

It is to be expected that their information requirements will have location as one of the primary relevancy filters. And while this may transpire in many ways throughout other verticals we have devoted plenty of time to figure out how it will influence travel decisions.

The Food Finder demo developed by the Geode team is a good example of how obvious the solutions will be. There is no need to elaborate more than that. Maybe hint the Hotel Finder, Theatre Finder, etc.

But knowing your latitude and longitude is only the first step; aggregating a good collection of trusted, relevant Web sites; properly geocoding all their entries so they can be checked for proximity against the user request, organizing all nearby candidates in a hierarchy that properly reflects the user preferences, and, finally, doing this for every single request in under a second requires quite a bit of technology.

Fortunately, this is exactly what we’ve been working on for the last few months and we are ready to deliver relevant, personalized travel recommendations.

If you want some serious restaurant recommendations…

In San Francisco @ 37.79594, -122.40331 ? Bubble Lounge
In New York @ 40.76143, -73.98227 ? Le Bernardin

Or Discover destinations around the corner or across the world.

Technorati Tags: , ,

PlanetEye

This is a blog about travel, photography, technology and their convergence online. Visit PlanetEye to learn more about our one-stop travel planning service.

Subscribe


Awards


Categories

Archives

PlanetEye is proudly powered by WordPress.