JSON, or javaScript Object Notation, is a lightweight, data interchange format, it also seems to have become the defacto standard for serialising and deserialising information when talking to web based APIs such as Twitter and Foursquare.
So far so good, but here’s the thing, I DON’T CARE! When I’m using the API of a web based service I’m doing so because I’m trying to solve a business problem. In the case of Twitter, I might want to know who is talking about me and my product, and what are they saying. I may also want to save that data away in a database for future analysis. That’s the problem I want to solve, so I don’t care about JSON and don’t care about the database.
Luckily DevEpress’ ethos is to remove the pain of boilerplate or plumbing code from you and so we can use XPO and a third party JSON library, in this case JSON.net, to allow us to concentrate on the problem in hand. So, because as developers we’re doers and not talkers, let’s stop talking about it and have a demonstration.
Let’s suppose we want to get a feel for the “word on the ground” from the current BUILD conference, Twitter is the natural place to look. There is a search API we can use so let’s try it out:
1: static void Main(string[] args)
2: {
3: string jsonString =
4: new WebClient().DownloadString(
5: @"http://search.twitter.com/search.json?q=bldwin");
6: }
Here we are using the Twitter API to search on the hashtag for BUILD. Putting in a breakpoint and inspecting the return shows what we get back from Twitter:
Yeah, ewww, I don’t want to have to deal with that. Time to recruit JSON.Net. Download the binary and reference it in you project and now we can turn the JSON into object, ‘cos we love objects, right?
Okay, so the first thing we have to do is to create an object for JSON.Net to deserialise this stuff into. Let’s do the simplest thing that works. Looking at the JSON above we can see that there is a wrapper object that contains the results. Let’s deal with that first. For reasons that’ll I’ll get into further on, we are going to be interested in the next_page property, so let’s build an object that can store that. Something like this…
1: namespace XPO_JSON_Blog
2: {
3: public class ResultsWrapper
4: {
5: public string Next_Page { get; set; }
6: }
7: }
Now, we can go ahead and deserialise the JSON, to this object, using JSON.Net…
1: using System;
2: using System.Collections.Generic;
3: using System.Linq;
4: using System.Text;
5: using System.Net;
6: using Newtonsoft.Json;
7:
8: namespace XPO_JSON_Blog
9: {
10: class Program
11: {
12: static void Main(string[] args)
13: {
14: string jsonString =
15: new WebClient().DownloadString(
16: @"http://search.twitter.com/search.json?q=bldwin");
17:
18: ResultsWrapper rw =
19: JsonConvert.DeserializeObject<ResultsWrapper>(jsonString);
20: }
21: }
22: }
That was pretty simple. Now if we put in a breakpoint and inspect the “rw” variable, we can see the following…
That’s better, now we are dealing with .Net objects in our .Net application, so we are happier. We can also see that we have this Next_Page property, what’s all that about? Well it turns out that Twitter isn’t going to give us all the Tweets at once, we’re going to get them in pages of 15 (by default) and we’ll also get the query string to execute to get the next page. Meh, that sounds like a faff.
There are a few things we have to do here. Firstly, we need to store the Tweets, after all that is the problem we are trying to solve, and to do that, first thing we need is an object to deserialise them into. For the purposes of this post, we’ll assume we are only interested in the sender of the Tweet, the date it was sent and it’s contents. So the object will look like this…
1: public class Tweet
2: {
3: public DateTime Created_At { get; set; }
4: public string From_User { get; set; }
5: public string Text { get; set; }
6: }
Oh, we mustn’t forget to add the collection of Tweet objects to the wrapper object so the deserialisation will work properly…
1: public class ResultsWrapper
2: {
3: public string Next_Page { get; set; }
4: public Tweet[] Results { get; set; }
5: }
Having done that, we need to make a few changes to our code. We need to add the “rpp=100” property to the querystring, that will tell Twitter to give us 100 Tweets at a time, instead of the default 15, that way we’ll need to make fewer calls. Then we need to separate the URL from the querystring, so that we can substitute the second and subsequent strings we’ll get from Twitter. After that we’ll need a function, which we can call recursively, to fetch the pages and some local storage for our Tweets. Phew, that’s quite a few changes, but really only a few lines of code. When we’ve done all that, our code will look like this…
1: class Program
2: {
3: private static List<Tweet> Tweets = new List<Tweet>();
4:
5: static void Main(string[] args)
6: {
7: string url = @"http://search.twitter.com/search.json";
8: string queryString = @"?q=bldwin&rpp=100";
9: FetchData(url, queryString);
10: }
11:
12: private static void FetchData(string url, string queryString)
13: {
14: string jsonString =
15: new WebClient().DownloadString(url + queryString);
16:
17: ResultsWrapper rw =
18: JsonConvert.DeserializeObject<ResultsWrapper>(jsonString);
19:
20: Tweets.AddRange(rw.Results);
21:
22: if (!String.IsNullOrEmpty(rw.Next_Page))
23: FetchData(url, rw.Next_Page);
24: }
25: }
Now, let’s see what we get if we pop in a breakpoint and inspect the “Tweets” variable…
Good! We’ve got a .Net object, with the properties we’re interested in, and we’ve abstracted away the JSON that we don’t want to deal with. Now onto the database!
Just as with the JSON, we want to abstract away the database, so we don’t have to worry about it and the easiest way to do that is to use XPO. Before we can start though we need to add references to the DevExpress.Xpo and DevExpress.Data libraries.
Having done that we can continue to add support for XPO to our objects. First, we’ll inherit from XpObject, and we’ll add the constructor taking a Session, or a derived class. then we’ll decorate the required properties with the size attribute, just to ensure there’s enough storage for them on the database. Here’s the Tweet class after we’ve done that…
1: public class Tweet: XPObject
2: {
3: public Tweet(Session session): base(session){}
4:
5: public DateTime Created_At { get; set; }
6: public string From_User { get; set; }
7:
8: [Size(140)]
9: public string Text { get; set; }
10: }
And that’s all we have to do to add XPO support to our project, pretty simple eh? Sadly, we’re not done just yet though, because things are a little more complicated on the JSON.Net end. You see we have to construct our XPO derived classes using a Session but, by default, JSON.Net has no idea how to do that so we have to give it a little hint, and we do that in the shape of a CustomerCreationConverter, in fact, we need one for each of our objects. The one for the Tweet object looks like this…
1: public class TweetConverter : CustomCreationConverter<Tweet>
2: {
3: private UnitOfWork uow;
4: public TweetConverter(UnitOfWork uow)
5: {
6: this.uow = uow;
7: }
8:
9: public override Tweet Create(Type objectType)
10: {
11: return new Tweet(uow);
12: }
13: }
And of course we need one for the ResultsWrapper class too.
Now all we have to do is to amend our application to use the new converters and a UnitOfWork…
1: private static void FetchData(string url, string queryString, UnitOfWork uow)
2: {
3: UnitOfWork theUnitOfWork;
4: if (uow == null)
5: {
6: theUnitOfWork = new UnitOfWork();
7: theUnitOfWork.ConnectionString =
8: "Data Source=.;Initial Catalog=BlogPostDB;Integrated Security=SSPI;";
9: }
10: else {theUnitOfWork = uow; }
11:
12: string jsonString =
13: new WebClient().DownloadString(url + queryString);
14:
15: ResultsWrapper rw =
16: JsonConvert.DeserializeObject<ResultsWrapper>(jsonString,
17: new JsonConverter[]{
18: new ResultsWrapperConverter(theUnitOfWork),
19: new TweetConverter(theUnitOfWork)
20: });
21:
22: Tweets.AddRange(rw.Results);
23:
24: if (!String.IsNullOrEmpty(rw.Next_Page))
25: FetchData(url, rw.Next_Page, theUnitOfWork);
26:
27: theUnitOfWork.CommitChanges();
28: }
There’s just one more thing we should do before we run our application and that is put in a little protection around the API call. We don’t really need to do much in this demonstration, we’ll just say, hey if the Twitter API returns any kind of error then we’ll just just finish there and store what we have. Something really simple like this will suffice…
1: private static void FetchData(string url, string queryString, UnitOfWork uow)
2: {
3: UnitOfWork theUnitOfWork;
4: if (uow == null)
5: {
6: theUnitOfWork = new UnitOfWork();
7: theUnitOfWork.ConnectionString =
8: "Data Source=.;Initial Catalog=BlogPostDB;Integrated Security=SSPI;";
9: }
10: else {theUnitOfWork = uow; }
11:
12: string jsonString;
13: try
14: {
15: jsonString =
16: new WebClient().DownloadString(url + queryString);
17: }
18: catch (WebException we){return;}
19:
20: ResultsWrapper rw =
21: JsonConvert.DeserializeObject<ResultsWrapper>(jsonString,
22: new JsonConverter[]{
23: new ResultsWrapperConverter(theUnitOfWork),
24: new TweetConverter(theUnitOfWork)
25: });
26:
27: Tweets.AddRange(rw.Results);
28:
29: if (!String.IsNullOrEmpty(rw.Next_Page))
30: FetchData(url, rw.Next_Page, theUnitOfWork);
31:
32: theUnitOfWork.CommitChanges();
33: }
Then we can run our application and store the search results in the database…
Of course, once we have them in the database we can hook up DevExpress Analytics and create some interesting visualisations. Hmm, now there’s an idea for my next post, ‘til then, happy coding! 
Update: @snowcode has been in touch to say instead of hand cranking your own classes you can use json2Csharp. Beware though that Json2CSharp won’t generate classes that are not in the original return from the API. For example, if your original return does not contain a Geo object then no Geo class will be generated. Subsequently, the Geo object will not then be deserialised even if future returns do contain a Geo object.