Category: Landscape

Or: privatesquare, ten months later

It would be generous to say that privatesquare was ever released. Instead, the source code was made public with a long and twisty blog post and a flurry of screenshots. I’ve told a few people about my installation of privatesquare but since I haven’t had the stamina to run a service for strangers (and a free one, at that) it’s remained small and reserved for friends and family. A few people along the way have set up their own instances and that’s been both gratifying and enormously useful in working out some of the kinks.

Maybe there are lots of copies of privatesquare running in the wild. I have no idea and no way of knowing. This is not a bad thing. In the meantime, I’ve continued to chip away at the project adding small features and fixing bugs as they come up.

Recently, we moved from San Francisco to New York City. Maybe foursquare, itself, woulda-coulda-shoulda started in another city but once you start to live the day-to-day ritual in New York it becomes pretty clear why it started here. The Atlas of Desire feature, described below, probably would have languished in a pit of good intentions if we hadn’t moved here but I’ll come back to that in a bit.

When it was first announced, last November, privatesquare came with a long list of tricks it hadn’t learned so it’s probably long overdue for a status update.

New old things

Here’s an unordered list of the things privatesquare said it didn’t do went I pushed it out of the nest:

• History pages

  

  There are now history pages for venues and dates (and date ranges) as well as for cities. Details on that below.

• Sync with the foursquare API…

  This is now possible, although it’s not enabled by default.

  Specifically, there is a little script that can be run by hand to import any foursquare check-ins that may have happened outside of privatesquare. This includes the history of check-ins that predate someone starting to use privatesquare. Or not. That’s why the feature needs to be explicitly enabled by each user choosing from one of three options:

  • Do not sync foursquare check-ins – Which is what it says on the tin. What you do elsewhere is entirely your business and privatesquare doesn’t need to know about it.
  • Only sync recent foursquare check-ins – Tell privatesquare to archive missing check-ins but only those that have happened since you signed up (with privatesquare).
  • Sync all foursquare check-ins past and future – This tells privatesquare to archive your entire foursquare history. This is what I’d expect most users to choose but it just always ends better when you let people make that decision for themselves.

• The nearest linear cell-tower problem…

  Sort of. Not really, though.

  The nearest linear cell-tower problem is caused by the fact that when a person is using privatesquare from their phone they are connected to the Internet through a cellular rather than a wireless network. Since the nearest cell-tower is used as a proxy for your location and since cell-towers have a range that is sometimes greater than the radius used to query nearby venues the list of places that gets returned is sometimes weird and full of bunk.

  One issue is that the W3C geolocation API, used by web browsers to determine your location, doesn’t tell you how your location was acquired. Was it from a wireless network? A cell-tower? A passing drone, overhead? There’s no way to know and by extension no way to adjust expectations.

  The nearest linear cell-tower problem is ultimately a user-interface problem and not one that’s ever been tackled particularly well. In more recent versions of privatesquare there is sometimes a little grey dot (rather than blue) that’s meant to indicate where the code thinks you are and help provide some context for the other dots.

  Another option is to allow you to grow the search radius for each query but I haven’t done that yet. File under: Something, however inadequate, instead of nothing.

• There is no ability to delete (or undo) check-ins…

  Done.

  

  Except for the part where there’s no way to delete check-ins using foursquare API which is … puzzling, to say the least. So in those instances where you’ve checked in to both privatesquare and foursquare and then delete check-in using privatesquare you’ll be presented with a short (exasperated) note explaining the situation and a link to the foursquare site where you can finish things up.

  I know, right?

• There is no ability to add new venues….

  Nope. Still not possible. Chances of it happening remain close to zero.

• Nor is there any way to record events offline…

  Sort of. Kind of. But only one particular facet of the problem is handled well.

  I first started thinking about this at SXSW this year because the combination of both the cellular network and foursquare being simultaneously overloaded with eager users meant that privatesquare was often unusable in downtown Austin.

  

  There are two problems that need to be sorted here:

  • The first is some way to store a check-in for future use when foursquare API is sad.
  • The second is some way to store a check-in for future use when the network itself is sad.

  The first problem is now accounted for with pending check-ins, something I’ll discuss more in a bit. The second problem isn’t really accounted for at all and at some point I just need to sit down and look carefully at what the Lanyrd kids did for their mobile site and probably just copy that.

• There is no way to distinguish duplicate venues (same name, different ID)…

  Nope. Still punting on that.

• Pages for venues…

  Totally!

  Every venue you’ve checked in to now has its own page on privatesquare. Each venue page has a list of the dates you’ve checked in there (including links to the actual check-in page), a link to other places you’ve checked in to nearby and a handy I am here now button for … well, check-ing in again.

  

  Designing for thumbs, and all that.

• Pages for cities…

  Yes!

  All check-ins in privatesquare are reverse-geocoded using the Flickr API which returns a list of Where on Earth (WOE) ID associated with a venue. At the moment only cities are recorded so you can’t filter check-ins by neighbourhood or country. Technically, there’s nothing to prevent it from being possible but cities seemed like a good and easy place to start.

  The places pages are kind of great and they are especially striking when you see them on a screen that is bigger than your phone. And because it uses the Flickr API it means that airports are bucketed as cities which makes even more sense in a foursquare context.

  

  Neighbourhoods are also a near-certainty given that no one’s geo infrastructure can figure out how to make peace with the meta-neighbourcities, called boroughs, that form the NYC metropolitan area and I actually have to suffer the consequences now that I live here. See also: the Atlas of Desire, below.

• Export…

  Yes!

  There are handy links at the bottom of most pages to export your check-ins as either a CSV or a GeoJSON file.

  Exports are available for the following buckets: check-ins by date or date range; check-ins for a venue; check-ins for a place. Exports should be available but aren’t yet for for: check-ins by nearby-iness; check-ins by desire (more below).

  It has also been the case that if you’re running privatesquare in a controlled environment (like a shared web-hosting service) that enforces its own usage limits exporting check-ins for a user with billions und billions und billions of check-ins can be problematic. I don’t have a ready-made solution for this problem, yet.

New new things

At the same time all of that was being done, there were a bunch of smaller things happening. In no particular order they are:

• Documentation!

  Gary Gale has done an amazing job of peeling away, with fresh eyes, the stack behind privatesquare and Flamework and keeping detailed notes about what everything means and how to get privatesquare up and running with a minimum of fuss in a variety of environments.

  Gary is also one of those people who checks in every time he exhales so he’s been excellent at finding the crumbly edge of things in privatesquare.

• Weather tracking

  

  privatesquare tries to record the weather (based a venue’s latitude and longitude) whenever you check in. Not much is done with that data except to show in the list of check-ins for a venue. The data is stored alongside the check-in so there’s always to opportunity to do something clever and interesting with in the future and in the meantime it makes for a nice soundtrack when scrolling backwards through the history of a place.

• Time of day tracking

  Sort of.

  Vladimir Agafonkin wrote a lovely little Javascript library to calculate the time pie for a given timestamp and Tom Carden was good enough to create a handy HTTP pony wrapper for it on Heroku that can be poked when a user checks in.

  All your check-ins that happened during the golden hour and that sort of thing.

  Unfortunately, with all the other network requests going out to third parties (foursquare, weather services and so on) the API call to get a time pie is often the one that fails or is killed because it happens so late in the chain of events. For the time being it is not a feature that is enabled by default. There’s a branch of privatesquare with the start of a pure PHP port of the code but it is both incomplete and full of bugs.

  If you’re not familiar with the idea of time pies, you should definitely read this blog post about them from the Stamen kids.

• Deferred check-ins

  Occasionally the foursquare API is sad. Sometimes it happens when you’re trying to use privatesquare. In the past you were sorry-out-of-luck but now when that first API call to fetch nearby venues fails privatesquare will give you the option of writing down the name of the venue you were trying to check in to and stick it, along with a timestamp, in your browser’s local (storage) cache.

  Once that happens a link titled pending will magically appears in the navigation menu and from there you can go back in time (sort of) and finish check-ing in.

  Essentially all that’s happening is the usual check-in process being replayed but with a canned search term and a note to privatesquare to use a an equally canned check-in time instead of of right now.

  

  For a bunch of perfectly good reasons you can’t do the same in foursquare so any pending, or deferred, check-ins are not forwarded along to the mothership. I may change that in the future and handle the spacetime disconnect by simply adding a note to the foursquare check-in. We’ll see.

  This is basically what should happen when the network itself is down or unavailable. Cities like New York are strange beasts that way. It’s hard to keep the scale of the infrastructure, and the burden of its history, in mind some days. Aside from the fact there are whole pockets of the city with terrible network (read: cellular) connectivity there are others that completely silent, to this day.

  Subways are an obvious place to want to check-in and privatesquare shouldn’t let itself be defeated by the dark spaces underground so teaching the code to play nicely as an offline application is rapidly bubbling up the totem pole.

  I’m told that if you have one of the fancy auto-refilling monthly New York City subway passes it’s possible to retrieve a history of all the stations where it’s been swiped so that bodes well for some sexy Clipper Futures style integration with privatesquare in the not too distant future.

• The Atlas of Desire

  

  Let’s be honest. On its current trajectory privatesquare will, sooner or later, become Dopplr. I don’t know what that says about foursquare and since the thing I built is, by design, not social I couldn’t exactly call the Chris Heathcote buckets described in the first blog post a Social Atlas. So, I called it what it is: an Atlas of Desire. Or, in concrete terms:

  • Web pages and list view for all the places you’ve checked-in to rolled up by the status you assigned: I am here; I want to go there; again again; and so on.
  • The ability to change a status after you’ve checked-in. For example, you could check-in to a restaurant when you sit down to eat and then change to again maybe when you leave.

  I’ve also added a new status simply called meh which, to my mind, is somewhere between again maybe and again never but ultimately the semantics are left up to individual users.

  If you asked me I’d tell you it’s for those places that you would visit for purely mechanical reasons: No matter how lackluster the food is it remains less bad than the bad craziness and temper tantrums caused when you melt down out of hunger.

  

  Shelley Bernstein, from the Brooklyn Museum, deserves a proper shout-out for the Idea of Meh which was part of her work making public the museum’s collection metadata.

  In addition to filtering your check-ins by status, you can further prune those places by nearby-iness or by city. It’s become pretty obvious that you should also be able to scope your desire to a neighbourhood. (See above inre: neighbourhoods.) It’s not a hard feature to add but it is a lot of typing so it will happen sometime between this blog post and re-wiring everything to work offline.

Still broken things:

Things that should never have been broken in the first place and that maybe I should fix before I say anything else, but oh well…

• OMGWTFTZ… timezones

  There are no excuses for this. It’s a thing that conveniently Just Works ® on the West Coast (because that’s the default timezone in the source code) and a thing complicated enough, under the surface, that I’d like to spend a little bit of time thinking about it before I go charging in to fix things.

  Probably the best thing would be to check for the timezone in the foursquare API response when you check-in and store that information locally. Failing timezone information in the API response I might build a simple httpony service on top of the whereonearth-timezeone dataset to do reverse geocoding for timezones.

  Sorry.

Things still TO DO

Mike Migurski has requested that privatesquare grow an I’m on my way status flag which is tempting but we’ll have to see about that one. Meanwhile, in no particular order these are some of the outstanding known-knowns:

• Make the tasteful pale grey permalink button and link for individual check-ins less invisible

  Yes, there are tasteful pale grey permalink buttons and links and for every check-in.

• Build scripts for “cloud app” hosting

  Gary’s documentation for setting up privatesquare is great and makes the whole process less painful but enough technical hoop-jumping remains to make it all inaccessible to a lot of people. Meanwhile, there are a whole raft of hosted services, like Heroku or PHPFog, that allow you to one-button install web applications which seem like they would be a good fit for things like privatesquare.

  Unfortunately the way the privatesquare is bundled (read: the ways the files and folders are organized) doesn’t lend it to one-button anything with many of these services. privatesquare (and similar tools I’ve been tinkering with at the same time) has never had versioned releases but it might be time be time to start freeze-drying the code at particular moments and creating discrete bundles targeted at specific environments.

  Those releases would always lag a little behind the new and shiny but the up-side is that it would be easier to install, which is probably a trade-off some people would make.

• Sharing. Maybe? Probably not…

  privatesquare is so-called for a reason. Every now and then I think about changing the default check-in status from don’t tell foursquare to don’t tell anyone which would, once you pulled in a user’s contact list, allow for sharing of things on privatesquare.

  I am still unconvinced about this one as it adds another layer of complexity and it’s not clear why it’s useful (foursquare is pretty good at that side of things) outside of the Atlas of Desire for which plain vanilla exports are probably more important. To whit:

• Better integration and/or pillaging of the Dotspotting codebase

  A couple months ago I forked the source code for Dotspotting and started to add the ability to edit individual dots. That’s really as far as I got and I’m not entirely sure where it all fits with privatesquare but the shadows of the two projects seem to overlap more often than not.

  Something something something see above something about exports something something something maps something something something.

• A proper API

  privatesquare already uses its own API but punts entirely on delegated authentication and still uses cookies for authentication and authorization. This is not ideal or, rather, is not a reason to lack an API that third-parties could build on.

  This is just another one of those tasks that requires some time where I sit down and spend the time to finish all the typing necessary to make it happen. Unfortunately, it is a thing made more complicated by the fact that having to read any of the OAuth specs makes me a little crazy in the head. It will happen.

• Notes

  Maybe. I’m of two minds which is to say I’m not inclined to add notes because I like the simplicity of the site, now, but could be convinced. For a while I had a version of privatesquare that would let you add notes to Findery (née Pinwheel) from venue, or check-in, pages and relied on the magic of machine tags (on their side) to stitch everything back together. I liked that and when there is a public API for the site (in the past I was… well, yeah, anyway…) I will probably slip it back in as an optional feature.

• Artisanal Integers!

  No, really.

  Privatesquare is at least half the reason that artisanal integers exist at all so it’s only fitting that is should start using them. Dan Catt’s blog post about London Integers is good place to start if you’re sitting there oscillating between confusion and table-pounding muttering the words artisanal… integers…

  Really.

One of the nicest things anyone has said to me about privatesquare since I started building it was: Oh, I’d love to use it but I can’t because I have an Android phone. The nice part is: you can! privatesquare is nothing more than a what-do-you-call-it web application.

The whole native versus web application debate on mobile is interesting, but only insofar as there are still a few tricks that web browsers can’t do or that they are prevented from doing by the operating system itself. Beyond that it all smells too much like the insane and self-serving hair-splitting that people, in the 90s and early 2000s, made about WAP and other uniquely mobile technologies. Read: Small computers, in your pocket.

privatesquare is meant to play equally well with your desktop as it is your phone. As time permits and I can work through what a print-specific CSS file looks like it should also work on that crazy retrograde technology that refuses to die: paper. The extent to which privatesquare fails, in any of those arenas, is more a reflection of (my) poor design skills than it is proof that somebody’s preferred form factor wins at celebrity death match.

And on and on, it goes!