TwitterSearch in Windows Phone: How hard can it be?

Some of you probably already know this, other probably don't care. But I am a Bruce Springsteen fan. A huge fan. I adore the man's music, I know the tunes, I love the music and I try to spend as much time as possible by playing his songs on my guitar (and fail miserably but having lots of fun doing so). I visit as many shows as I can or as I can afford. So when Bruce decided to have another tour in the year of 2014 I got a little nervous. His shows usually sell out in a couple minutes which means that as soon as the shows are announced you have to be ready to buy tickets. That means that you have to know what's going on in Springsteen-land and for me, that meant I had to write an application for my phone that tells me exactly these sort of thing. It didn't seem that hard to do: just write an app that reads out some RSS feeds and show them on the screen.

But then I thought a little bit harder about it and thought that maybe other people might be interested in this as well. Of course, that meant that the competition for those tickets would be a bit tougher but what the heck, as Bruce himself said, nobody wins as everybody wins.

Now, writing a simple app just for myself is one thing, writing an app that gets downloaded by people all over the world is something else. If it were only for myself I could have finished the thing in a couple of hours, but publishing an app for real means I had to seriously think about it and make sure it would be stable, nice-looking and useful. Which means I had to add features I didn't need just for keeping up with the latest show announcements.

The idea behind writing apps is that you ship early and ship often, so the first version was out of the doors pretty quickly. It did little more than just displaying a list of the last 40 news items on the web that were about Springsteen. The second version was a bit more complicated: it had a live tile telling you exactly how many new news items were available for you to read since the last time you started the application. This uses Windows Azure Mobile Services as the backend so it gave me the opportunity to learn how to use that as well. After all: I would be using this app to learn new techniques as well as giving me the information I wanted.

When that version was finished and published I started thinking about new features. One of the features I thought would be nice to have is an overview of Tweets that contain the word "Springsteen". After all: when there is news that hasn't been really published yet, people will talk about it on Twitter. So I had to add another news source (on another screen) that contains the results of a Twitter search.

Since Twitter apps are the new "Hello World" apps I figured it couldn't be hard to build. And if I were to run into problems I would find documentation and samples all over the web, showing me the way. Boy, was I wrong.

Of course, there are a lot of sites and articles that tell you how to do this. But these articles have two problems:

  1. They are written for Windows Phone 7. Things have changed since then.
  2. They are written for Twitter API v1. Things have changed since then.

In fact, things have changed so much I couldn't use the information out there at all. So I had to figure it out all myself. Well, I did just that and I am presenting you with the way to do this right here so you don't have to worry about it.

Twitter side of things

First, the biggest change on the Twitter side of things is that in the old days you could go out to http://api.twitter.com/search and get the results you wanted. This doesn't work anymore. They want you to authorize your app first. So you have to log in.

Now, there are two ways you can do this. First you can ask your user to log in and then search on his behalf. This is not really acceptable in my case since all I want to do is show the search results. I don't need the users credentials: I won't be posting or re-tweeting anything. So I have to use the second option which is let the application log in for the user. This means the app has read-only access to the Twitter API but that's just fine in this use case.

To do this, you have to register your app with Twitter. Go to https://dev.twitter.com, sign in and reserve your app name. If you do that, you get some keys that you're going to need. For my app BruceNews it looks like this:

I have edited out the keys but it's just a bunch of characters you get when you let your cat walk on your keyboard for a while. These keys, the API key and API secret are what you need to authenticate your requests. The reason for this feature is that while the usage of Twitter is free, they don't want you to overdo things. There is a limit to the number of requests you can fire against their servers. At the time of writing, this is 450 calls per 15 minutes for the search, per app instance. So that's once every two seconds which is reasonable enough for our app. I don't expect users to refresh the search more than that for fifteen minutes at the time….

When you register your app they also want you to give a call back URL that gets called when users use your app to authenticate themselves. However, you don't need that so you can fill in whatever you want.

Now, for an app to do a Twitter search, the flow is as follows. First, you authenticate your app against Twitter. They will give back a token key. This token key is then passed on to Twitter again whenever you do an API call. Remember: this authentication scheme only allows you to do read-only calls such as search!

Windows Phone 8 side of things

It's time to start up Visual Studio and create a new Windows Phone 8 app. I have made a very simple test application, so all I needed was the standard Blank App template. I won't be using MVVM here since I am only showing you how to use the Twitter Search. Of course, if you do want to create a real app I strongly, really, really strongly urge you to use an MVVM framework. For now, the MVVMLight stuff by Laurent Bugnion is your weapon of choice for both Windows Phone 8 as Windows 8 Store Apps. But I digress.

My main screen looks pretty simple. All it does is show a Listbox. In that Listbox I have added a datatemplate for the itemtemplate to format the tweets. Twitter has some guidelines to how those things should look like. They really want all tweets to look the same:

However, there is a nice little loophole: the developer side says: "Applications that cannot implement these guidelines due to the technical limitations of their platform must make a best effort to satisfy as many rules as possible" Since we want our tweets to adhere to the Windows Phone look and feel, we can more or less deviate from this standard. Still, it's a good idea to implement as much of this as possible since it makes it for users easy to recognize as being a tweet. I use the avatar, screen name, handle, text and postdate in my example.

So, how do we get the data from the Twitter service? Well, you need to add a Nuget package first. All data from Twitter is in JSON format so you need a way to handle that. There is a more or less standard package which helps for that: NewtonSoft JSon.net. You can install through the "Manage NuGet Pacakges…" context menu in the solution explorer or through the Package Manager Console and type in "Install-Package Newtonsoft.Json". Another package you will need is the Microsoft HTTP Client Libraries (Install-Package Microsoft.Net.Http). Now, you're good to go.

I have defined two static members in my codefile, named TwitterConsumerKey and TwitterConsumerSecret, which are the two keys we got from Twitter. Next to that, I have added a Loaded event handler on my main screen that does two things: Load the tweets and assign the resulting list to the listbox on my main screen.

Now, before we go to the nitty-gritty details of how to get the data out of the Twitter server, we need to have classes that hold the tweets. Now, don't you think a tweet is just a simple set of text lines. The structure you get back for a single tweet is rather impressive:

Don't try to read it. It's hard to read since it's pretty small in this image. But it does give you an idea of the complexity. Especially the user is quite elaborate.

On dev.twitter.com you can get the resulting JSON and there are tools out there that can help you transform this JSON result into C# classes. You need to do that so you can actually use them in your Phone application.

I have added these classes to my project and now I am ready to begin reading the data. Remember, there are two steps to take: first authenticate, then get the results.

First, let's do the authentication.

Authenticate against the Twitter REST API.

If you look at the steps on the Twitter site, it seems pretty straightforward. However, to translate this into valid code for a Windows Phone 8 app you need to go through some hoops. Here it is:

private async Task<string> GetAuthenticationToken()

{

var client = new HttpClient();

var uri = new Uri("https://api.twitter.com/oauth2/token");

 

var encodedConsumerKey = HttpUtility.UrlEncode(TwitterConsumerKey);

var encodedConsumerSecret = HttpUtility.UrlEncode(TwitterConsumerSecret);

var combinedKeys = String.Format("{0}:{1}", encodedConsumerKey, encodedConsumerSecret);

 

var utfBytes = System.Text.Encoding.UTF8.GetBytes(combinedKeys);

var encodedString = Convert.ToBase64String(utfBytes);

client.DefaultRequestHeaders.Add("Authorization", string.Format("Basic {0}", encodedString));

 

var data = new List<KeyValuePair<string, string>>

{

new KeyValuePair<string, string>("grant_type",

"client_credentials")

};

var postData = new FormUrlEncodedContent(data);

 

var response = await client.PostAsync(uri, postData);

if(response.StatusCode != HttpStatusCode.OK)

throw new Exception("Did not work!");

 

var content= await response.Content.ReadAsStringAsync();

var authenticationResponse = JsonConvert.DeserializeObject<AuthenticationResponse>(content);

if(authenticationResponse.TokenType != "bearer")

throw new Exception("wrong result type");

 

return authenticationResponse.AccessToken;

}

}

 

Let me walk you through the code. First of all, you need to mark this method as async. After all, it might take more than 50 milliseconds to authenticate and the guidelines say you need to make it asynchronous if it takes that long. So mark the method with async and let it return a Task<string>. The string in this case is the authentication token I was talking about earlier on.

Now, create an instance of the HttpClient. This comes from the HttpClient package we installed earlier on. The uri I give is the one you need for authenticating, so that's pretty straightforward as well. Now, the next few lines prepare the two secret keys in a way Twitter expects them. First I UrlEncode them. Twitter suggests I do this. In reality this doesn't do anything since the keys do not contain any characters that might need escaping, but the Twitter documentation says this might change. So I decided to do this. Better safe than sorry, right? After this encoding I combine the two strings with a colon (:) in between. I get one string as a result. This string needs to be passed on to Twitter but it needs to be Base64 encoded. Since you can't directly do that in Windows Phone, I first need to translate the string into an array of bytes which then in turn get turned into a Base64 string.

Now, the tricky part. The API wants this key to be in the headers of the request. So I add to the DefaultRequestHeaders a value of this string with the key being "Authorization". It also wants some extra data in the form of the key-value pair "grant-type" / "client_credentials" so I make a list of this one key-value pair, give it to a new FormUrlEncodedContent which I pass on to the actual request I make: client.PostAsync.

Once you do that the Twitter authentication mechanism starts to work and will get back to your. It will do that with either a response 200 (which means it all went ok) or another one saying something went wrong and that you probably are the one to blame. And they usually are right, darn them…

Let's assume it all went ok and now it's time to look at what we got back. The response object contains a Content stream that we can read out with ReadAsStringAsync(). This content can be converted to an object of type AuthenticationResponse. This class I created myself and looks like this:

class AuthenticationResponse

{

[JsonProperty("token_type")]

public string TokenType { get; set; }

[JsonProperty("access_token")]

public string AccessToken { get; set; }

}

 

It's just a placeholder for two fields, namely token_type and access_token. I have decorated the fields with the correct Json names so I can use the right format for the property names and thus prevent my tooling for complaining about my crappy naming convention. Now that I think about it, I did the same for the Twitter classes….

If all went well we now have an instance of the AuthenticationResponse. This can contain several things, but if things worked as they should you now should have an TokenType (or token_type if you use the original name) of "bearer". The value belonging to this field, stored in AccessToken (or access_token) is what you need to pass on to the next call. If this is not "bearer" you're in trouble and you did something wrong…

Let's do some searching!

Now that we have this authentication token we can do the actual search. If you understood the last part, you will find the search is pretty simple. It does things more or less the same way. Let me show you my code. First I have created a wrapper object which will contain all search results:

internal class TwitterSearchResults

{

[JsonProperty("statuses")]

public List<RootObject> Statuses { get; set; }

}

 

This is what we will get back from the search. It's just an object with a list of RootObject instances. This RootObject you can find in the diagram above (if you look really hard) but that contains information about the user, the tweet itself, the time and date and so on.

The search looks like this:

private async Task<List<RootObject>> GetTwitterResults()

{

string token = await GetAuthenticationToken();

var client = new HttpClient();

var uri = new Uri("https://api.twitter.com/1.1/search/tweets.json?q=%23springsteen&count=30");

client.DefaultRequestHeaders.Add("Authorization", string.Format("Bearer {0}", token));

 

HttpResponseMessage response = await client.GetAsync(uri);

Task<string> content = response.Content.ReadAsStringAsync();

var searchResult = JsonConvert.DeserializeObject<TwitterSearchResults>(content.Result);

 

return searchResult.Statuses;

}

 

Again, we need to mark this method with async. The result is a Task<List<RootObject>> so in effect we get the list with tweets we can databind against. The first thing I do is call our previously discussed GetAuthenticationToken. After all, each call to the API needs this token otherwise it just will not work.

The moment we have this token we create a new instance of the HttpClient. Then we build up the query we want to perform. I have hardcoded the search string "#springsteen" (the hashtag is UrlEncoded otherwise it won't work) and instead of the default 15 results I want to get back the last 30 results. There are about 30 options you can specify here. You can limit the results to a certain date, or a certain geographical location and so on but I would suggest you keep things simple at first. First make it work, then make it pretty, remember?

Just like we did in the authentication code we add a new header, but this time we give it the key value pair "Authentication" – "Bearer xxxx" where xxxx stands for the token we just got back.

Now all we have to do is call client.GetAsync(uri) (we did a Post in the authentication, this requires a Get) and again we get back a response. You really should check for the code again, it should be 200, but for brevity I decided to "forget" about that here. I assume it all just works…

The resulting string in the Content is deserialized to a TwitterSearchResults object and returned to the calling Loaded event handler. This then puts that data into the Itemsource for the listbox and voila, there it is! The tweets will show up on the screen.

Conclusion

It's really not hard to get Twitter search results in your Windows Phone 8 app, but the lack of documentation and samples make it quite a challenge to find out how to do this exactly. But once you find out how to do this, you can enrich your app with real time results matching your audience. And that makes an app a lot more useful for a lot of people. At least, I know now that there are no plans for an European tour so I don't have to worry about getting tickets… yet.. wait, let me check again…

 

 

 

Print | posted @ Sunday, March 23, 2014 7:31 PM

Comments on this entry:

No comments posted yet.

Post A Comment
Title:
Name:
Email:
Comment:
Verification: