XPO

Using SQL Server 2008 spatial data from XPO

Oliver Sturm (Developer Express) — Tue, 19 Aug 2008 14:00:00 GMT

Boris posted me some code that I used to play around with, in order to make XPO work with SQL Server 2008 spatial data. It is possible to make it work, quite easily actually, but there are a few things that need to be considered, and where more than one solution is possible. Let's see.

First, we need a class to store data in the client application. This is the one I'm using:

  public class PolygonData: XPObject {
    public PolygonData(Session session)
      : base(session) { }
 
    private string name;
    public string Name {
      get { return name; }
      set { SetPropertyValue("Name", ref name, value); }
    }
 
    SqlGeography polygon;
    [ValueConverter(typeof(GeographyConverter))]
    [DbType("geography")]
    public SqlGeography Polygon {
      get { return polygon; }
      set { SetPropertyValue("Polygon", ref polygon, value); }
    }
  }

This is already the first thing that probably needs some thought. In order to use the SqlGeography class as the property type, it is necessary to have a reference to the assembly Microsoft.SqlServer.Types. That assembly is not currently part of the .NET Framework, but it is installed together with SQL Server. As the SqlGeography class has several pieces of useful functionality (and there are other useful classes in the same assembly), it seems weird that they aren't available in the "standard client". Apparently Microsoft prepared these classes for server-side use, and obviously this makes some sense if you're going to use .NET on the server in some way, but still... there are a number of blog posts out there on the topic (try searching SqlGeography client side, for instance), and at least one of them points to a future solution that involves a separate redistributable which might become available at some point. For the time being, it would probably be possible to install the assembly in question on the client manually -- but please note that I haven't looked into legal issues involved with this, so be careful!

If, for whatever reasons, you don't want to go this way, you would have to create your own client-side class to represent the spatial information. The other approaches described in this post would still be quite similar though.

Now, as you can see, the property for the Polygon in my code is decorated with two attributes. The DbType attribute makes sure that the field gets created with the correct type "geography" in SQL Server. The ValueConverter attribute is used to convert the geography into the string format needed in SQL. Here it is:

  public class GeographyConverter : ValueConverter {
    public override object ConvertFromStorageType(object value) {
      if (value is string)
        return SqlGeography.Parse((string) value);
      else return value;
    }
 
    public override object ConvertToStorageType(object value) {
      return value == null ? null : ((SqlGeography) value).ToString( );
    }
 
    public override Type StorageType {
      get { return typeof(string); }
    }
  }

The ConvertFromStorageType function implementation is a bit weird, since it simply skips the conversion if the object that's getting passed in is not a string. This is perhaps not the most secure way of implementing this, but it's good enough for the purpose of the demo. The reason I'm expecting objects that might not be strings is explained further down and it has to do with performance -- depending on the decision you make at that point, you might implement this function in slightly different ways.

To create a bit test data, I'm using the following helper function in my code (the SqlGeographyBuilder is another class from Microsoft.SqlServer.Types):

    private static SqlGeography CreatePolygon( ) {
      SqlGeographyBuilder builder = new SqlGeographyBuilder( );
      builder.SetSrid(4326); 
      builder.BeginGeography(OpenGisGeographyType.Polygon);
      builder.BeginFigure(55.36728, -2.74941);
      builder.AddLine(55.40002, -2.68289);
      builder.AddLine(55.39908, -2.74913);
      builder.AddLine(55.36728, -2.74941);
      builder.EndFigure( );
      builder.EndGeography( );
      return builder.ConstructedGeography;
    }

With this in place, I can create and store some data:

      using (UnitOfWork uow = new UnitOfWork( )) {
        new PolygonData(uow) {
          Name = "Test 1",
          Polygon = CreatePolygon( )
        }.Save( );
        uow.CommitChanges( );
      }

If you are following along and you've built your own sample with these code snippets, you will be able to execute the sample at this point. It will get a table created in SQL Server and the spatial data inserted. Wonderful!

Finally, of course we want to read data back from the database. In my sample, I'm using this simple piece of code to do it:

      using (UnitOfWork uow = new UnitOfWork( )) {
        var polygons = new XPCollection<PolygonData>( );
        foreach (var polygon in polygons) {
          Console.WriteLine(polygon.Name);
          Console.WriteLine(polygon.Polygon);
        }
      }

If you try to run this, you will see an exception though. The reason for that is that there are types here which don't match up. The SQL Server client library, somewhat confusingly, returns an object that is actually of type SqlGeography, whereas our type converter defines a storage type of string ins StorageType property. As a result, a function called ReformatReadValue is called on the connection provider we're using (MSSqlConnectionProvider by default), and that function attempts to use the .NET Framework standard Convert.ChangeType function to convert a SqlGeography into a string. That function in turn expects the object to implement IConvertable, and throws an exception because it doesn't do that. Phew.

So what's the solution to this problem? Well, making the ReformatReadValue function do the conversion of SqlGeography into string, that's one solution. Let's derive a connection provider and override that function:

  public class GISProvider : MSSqlConnectionProvider {
    public GISProvider(IDbConnection connection, AutoCreateOption autoCreateOption)
      : base(connection, autoCreateOption) {
    }
 
    protected override object ReformatReadValue(object value, ReformatReadValueArgs args) {
      if (value != null) {
        Type valueType = value.GetType( );
        if (valueType == typeof(SqlGeography) || valueType == typeof(SqlGeometry))
          return value.ToString( );
      }
      return base.ReformatReadValue(value, args);
    }
  }

Now we need to make sure this is the provider we're using, instead of the standard MSSqlConnectionProvider. We can do this with an initialization line like this:

      XpoDefault.DataLayer =
        new SimpleDataLayer(new GISProvider(
          new SqlConnection("data source=.;integrated security=SSPI;initial catalog=XPOSql2008Spatial"), 
          AutoCreateOption.DatabaseAndSchema));

If you're following along, try running your application again, with the reading code in place, and you should see the geography information read back to the client and shown on the console.

There's one problem with this code, and I'm sure you have noticed already: we are getting back a SqlGeography instance from the database client code, and this gets passed in to the ReformatReadValue method in the XPO infrastructure. There it is converted into a string. Then it gets passed into the configured value converter for the Polygon property, and gets converted in a SqlGeography instance. Sounds suboptimal, doesn't it? Yeah...

There is a solution to this, but it's not entirely perfect. It is possible to simply ignore what ReformatReadValue wants us to do and just not convert the object (leave off the ToString() from the value the method returns). Then the SqlGeography type object will be passed in to the value converter, which needs to be implemented to ignore the fact that this is not really a string it receives. Remember, I already pointed this out above - if you want to go this "better performance" way, your value converter will have to be able to deal with the fact that it might receive objects to convert that have actually already been converted.

I'm only describing this solution instead of showing the code, because it's not really something I want to recommend. This solution neglects the contracts of the XPO infrastructure, and that is of course a bad thing when changes are made in the future. It might also be a problem if the data has to be serialized after being read from the database - having it in string format is what XPO normally assumes, and that works well with all types of serialization. I haven't made any real tests, so I can just say I don't know precisely what will happen if SqlGeography instances are sent across the wire using Remoting, XML Web Services, WCF or other frameworks. So - if you're interested in the performance gain, Remoting and so on are perhaps not a concern of yours (it seems unlikely anyway that Remoting and the perf gain from a few saved string conversions are *both* important to you), then it shouldn't be hard to use this approach I described. Please note though that you should take extra care to have relevant unit tests in place, so that changes in future XPO versions don't catch you out.

Overall it seems a bit weird the way this feature has been implemented. The client side seems to convert the spatial data into the original type automatically, pretty much whether you want it or not - at least that's how I understand it at this point. Since I push in string data through SQL in order to store the information, it would seem logical to be able to retrieve the string data only, and make my own decision about converting it back into object data, and specifically about the right point in time to do this. Perhaps it fits in somehow with Microsoft's decision not to make Microsoft.SqlServer.Types available in the client by default. Or perhaps there's something I missing right now in this regard :-)

Anyway, have fun! Here's the complete code of the sample I created: XPOSql2008Spatial.zip (4201 bytes)

XPO and FILESTREAM - update

Oliver Sturm (Developer Express) — Fri, 15 Aug 2008 10:40:00 GMT

Yesterday I wrote about using the new SQL Server 2008 FILESTREAM feature with XPO, and this morning Boris from our XPO team sent me an IM going "na na na, I found an easier way to do this". Okay, maybe he didn't quite put it like that :-) Anyway, here's the code he sent me:

  public class Something : XPObject {
    public Something(Session session)
      : base(session) { }
    public override void AfterConstruction( ) {
      base.AfterConstruction( );
      streamId = Guid.NewGuid( );
    }
    private string strVal;
    public string StrVal {
      get { return strVal; }
      set { SetPropertyValue("StrVal", ref strVal, value); }
    }
    private byte[] streamData;
    [DbType("VARBINARY(MAX) FILESTREAM")]
    public byte[] StreamData {
      get { return streamData; }
      set { SetPropertyValue("StreamData", ref streamData, value); }
    }
    private Guid streamId;
    [DbType("uniqueidentifier ROWGUIDCOL UNIQUE NOT")]
    public Guid StreamId {
      get { return streamId; }
      set { SetPropertyValue("StreamId", ref streamId, value); }
    }
  }

This code removes the need for any separate structural updates, which is of course very convenient. Now, in my previous post I had already mentioned I'd been playing with ways of using the DbTypeAttribute, and my solution had looked almost exactly the same - only I hadn't been aware that the UNIQUE keyword could be used in that precise way, and so I wasn't able to get the necessary index created on the ROWGUIDCOL field. Boris' code takes care of that now.

Is this preferable? I'm not entirely sure. I had one other reason to decide not to post my own similar code in my previous article, and that was the fact that the DbType parameter for the StreamId field looks very much like SQL injection. As you can see, it ends in "NOT", and that is to combine with the "NULL" that XPO inserts into the SQL code anyway, so the field ends up being "NOT NULL". Hm... I guess this is SQL injection, seeing how the text from the DbType is inserted into the DML code without any checks. Then again, that's the exact point of having a DbType attribute in the first place - the ability to use types that XPO isn't aware of - and of course it's compile time data that is injected. In the end, that code has the same effect as the other code I published previously. It is important to point out that while this may look similar to code used for SQL injection attacks, it's quite a different thing really, and there doesn't seem to be a way of exploiting this at runtime.

So is it preferable? I don't know - I guess it's more convenient to use in some cases. It's also less dynamic. With the code I published previously, it would be possible for the initialization code to determine whether SQL Server 2008 is being used, and to make the necessary changes only if it is. By decorating the field with the DbType attribute, the class can only be used with SQL Server 2008. I guess in the end everybody will have to decide for themselves.

Update: Another idea from one of our untiring XPO team members - Alex this time - you could use an XPCustomObject and establish a Guid type primary key. This will be marked with ROWGUIDCOL automatically (we introduced this originally to make the databases replication friendly) and of course it will have the required unique index. I actually had a similar idea early on, but I wasn't going to do this because I didn't want to mix the requirements of the FILESTREAM feature with other structure... Of course you'll still have to get the FILESTREAM type set correctly and using DbType on that one still makes your structure specific to SQL Server 2008. I guess my own overall decision might still be to make the modifications through SQL/DML statements, but of course (as Alex also pointed out) I should pay attention whether my table has a Guid type primary key perhaps - in which case SQL Server will not let me establish a second column with the ROWGUIDCOL attribute set.

Finally, I was asked to show how initialization code can be used in the context of an XAF application. That's actually very simple, since XAF has its own mechanism of updating database structure and content with new versions of applications. That mechanism can easily be used to call some structure modification code. It's documented in the XAF help here.

XPO and SQL Server 2008 FILESTREAM support

Oliver Sturm (Developer Express) — Thu, 14 Aug 2008 16:48:00 GMT

In a recent forum post, Reinhold Erlacher asked me whether we supported the new FILESTREAM feature in SQL Server 2008 yet. I was not familiar with the feature, not being a SQL Server expert really, but I decided to have a look.

First, I had to set up SQL Server to actually support the feature in question. A well documented process, but a bit weird nonetheless - why do they introduce features that are only available after jumping through so many additional hoops? Ah well... Here are the instructions on MSDN - there's a lot of confusing information around in blogs, apparently from pre-release versions of SQL Server 2008, so be sure to use these to get it to work with your database.

Now, first I played around a bit to try and get XPO to create a database structure automatically that would be compatible with the FILESTREAM feature. That is not currently possible, as it turns out, since there are a few requirements:

The database must have a file group for the file streams
A table that has associated file streams must also have a column marked ROWGUIDCOL
The ROWGUIDCOL column must have a unique index applied

Finally, of course, the blob column that will store the stream data must be marked FILESTREAM. We don't support any of these things directly - i.e. through a built-in feature - in XPO, and I don't think it's likely that we will. Fortunately, it's quite easy to make the necessary changes either manually (configuring the file group in SQL Server Management Studio) or automatically (using a bit of SQL code that's executed after the default schema creation process from the XPO application). The process shouldn't be hard to integrate with a typical deployment scenario, and of course we can't help the fact that Microsoft have made it quite hard to benefit from the new feature.

I'm using this basic persistent class for my demo:

  public class Something: XPObject {
    public Something(Session session)
      : base(session) { }
 
    public override void AfterConstruction( ) {
      base.AfterConstruction( );
      streamId = Guid.NewGuid( );
    }
 
    private string strVal;
    public string StrVal {
      get { return strVal; }
      set { SetPropertyValue("StrVal", ref strVal, value); }
    }
 
    private byte[] streamData;
    public byte[] StreamData {
      get { return streamData; }
      set { SetPropertyValue("StreamData", ref streamData, value); }
    }
 
    private Guid streamId;
    public Guid StreamId {
      get { return streamId; }
      set { SetPropertyValue("StreamId", ref streamId, value); }
    }
  }

I experimented briefly with the DbTypeAttribute and I had some success with it, but since custom SQL code was still required, I decided to make all relevant changes in one place instead of spreading them out. My test application is this:

  class Program {
    static void Main(string[] args) {
      XpoDefault.DataLayer = XpoDefault.GetDataLayer(
        MSSqlConnectionProvider.GetConnectionString(".", "XPOFileStream"),
        AutoCreateOption.DatabaseAndSchema);
 
      // This sets up the DB structure. Of course we would only do this 
      // once in a real app. Since we're boring in this sample, we're just
      // going to kill the db and go from scratch.
      using (UnitOfWork uow = new UnitOfWork( )) {
        uow.ClearDatabase( );
      }
      CreateDBStructure( );
 
      // Create some test data
      using (UnitOfWork uow = new UnitOfWork( )) {
        Something something = new Something(uow);
        something.StrVal = "Entry";
        something.StreamData = UTF8Encoding.Default.GetBytes("Stuff in my stream");
        uow.CommitChanges( );
      }
      // Read the object back and show the stream data
 
      using (UnitOfWork uow = new UnitOfWork( )) {
        Something something = uow.FindObject<Something>(null);
        Console.WriteLine("Stream says: " + UTF8Encoding.Default.GetString(something.StreamData));
      }
    }
 
    private static void CreateDBStructure( ) {
      // Get the basic db structure created
      using (UnitOfWork uow = new UnitOfWork( )) {
        uow.UpdateSchema(typeof(Something));
      }
 
      // Apply schema modifications
      Action<string> execute = CommandExecutor(XpoDefault.DataLayer.Connection);
      execute("alter table Something alter column StreamId uniqueidentifier not null");
      execute("alter table Something alter column StreamId add rowguidcol");
      execute("alter table Something add constraint streamid_unique unique(StreamId)");
      execute("alter table Something drop column StreamData");
      execute("alter table Something add StreamData varbinary(max) FILESTREAM");
    }
    static Action<string> CommandExecutor(IDbConnection connection) {
      return delegate(string command) {
        SqlCommand sqlCommand = new SqlCommand(command, (SqlConnection) connection);
        sqlCommand.ExecuteNonQuery( );
      };
    }
  }

You can download the complete sample from here (VS 2008 SP1): XPOSQL2008FileStream.zip (4267 bytes)

Note: If you try to run the sample out of the box, you will be in trouble because the database is not configured to support file streams! You should therefore create a database called "XPOFileStream" manually and set it up to support file streams correctly (again, as described here), and then run the sample.

XPO Publication Service and XAF

Oliver Sturm (Developer Express) — Thu, 14 Aug 2008 10:57:00 GMT

I was involved in some debugging action this morning, regarding my XPO Publication Service. Please read this post (and others linked from there) to find out the basics about that service.

The problem in question was that a customer was receiving a weird Remoting exception when trying to connect to a data store published through the publication service. The first test I did was to change the dependencies in the service to 8.2, build it, and run my functional tests on it - all of those worked fine.

So I set out to create a simple client that would attach itself to a data store published through the Publication Service. At this point, I think a little excursion would be good. I have noticed that many people have the weirdest ideas of how to connect to a published service from XPO. Often they involve a line like this:

IDataStore dataStore = (IDataStore)Activator.GetObject(typeof(IDataStore), 
  @"http://tcp:2635/something.rem");

Now, this is not wrong, but it's also not necessary. Retrieving a reference from the Activator is one way of interfacing with the .NET Remoting infrastructure on the client side. But - that code is already contained in the XPO libraries, so all the gory details of using Remoting are hidden. You can easily use XPO to get a connection provider (= data store) created directly, like this:

XpoDefault.GetConnectionProvider("tcp://localhost:9999/something.rem", 
  AutoCreateOption.DatabaseAndSchema)

Or, even better, you can get a data layer set up directly, like this:

XpoDefault.DataLayer = XpoDefault.GetDataLayer("tcp://localhost:9999/something.rem", 
  AutoCreateOption.DatabaseAndSchema);

What this means is really that a connection URL for a data store published through Remoting can be used in very similar places to any other connection string. XPO will figure it out for you.

On a related note, guess how to set up an XAF application to connect to a data store publication? That's right, just use the URL as a connection string. For instance, put this into your app.config:

<add name="ConnectionString" connectionString="tcp://localhost:9999/something.rem" />

Right, end of excursion. Now, I'd created that little client app and, in contrast to all the lines above, I was using HTTP, not TCP, for the connection. The publications.xml file for the Publication Service had this Channels section:

<Channels>
  <xpopub:HTTPTargetChannel>
    <Port>9999</Port>
  </xpopub:HTTPTargetChannel>
</Channels>

I was quite surprised to find this, but I got the same Remoting exception as described in the support center issue.

One idea I quickly had was that the problem might be related to that other weird issue I had encountered at some point when using HTTP for a publication. That issue is described in this post on my personal blog, and it's certainly not irrelevant if you use Remoting or some other protocol to publish things through HTTP. So go to this article and have a look if you're interested.

Anyway, that wasn't it, since the error messages were different. Nevertheless, I was able to solve the problem by switching to TCP. Probably a good idea anyway, unless you specifically need to use HTTP for firewall reasons (though it will forever remain a mystery to me why people don't have a problem tunneling unknown content through HTTP, while being scared $$(*&less by the same data on a different port). So I changed my publications.xml to this:

<Channels>
  <xpopub:TCPTargetChannel>
    <Port>9999</Port>
  </xpopub:TCPTargetChannel>
</Channels>

I changed all the URLs I was using to tcp://, as shown in the code lines above, and then it worked. Great. A quick test with the more complex XAF Main Demo application showed that this worked as well. Fine.

So, back to the "why"... why is everything fine with TCP, but not with HTTP? Well, the answer came to me after some discussion with our support guy Dennis: the serialization formats don't match. Yeah. Let me say that again - the serialization formats don't match. Why doesn't Remoting throw a TheSerializationFormatsDontMatchException? I don't know. Anyway, that's what's happening.

The XPO Publication Service defaults to binary encoding, even for the HTTP channel. There's a property called UseBinaryEncoding on the channel, which can be changed through the config file, but it defaults to true. The XPO library, on the other hand, defaults to SOAP encoding when an HTTP channel is used. So, back to the publications.xml with the HTTPTargetChannel, and then insert this line before the code that establishes the connection to the server:

ChannelServices.RegisterChannel(new HttpChannel(null, 
  new BinaryClientFormatterSinkProvider(), null), false);

Credit for this line goes to Dennis - he had found out that this makes things work, though he wasn't exactly sure why. Here's why: because the HTTP Channel is configured to use binary encoding and thereby becomes compatible with the way the Publication Service works. The XPO library doesn't register the HTTP Channel again, since it's already there, and it doesn't change the registration either. Fine.

Now, several questions come up here. First, why are the Publication Service and the XPO defaults incompatible? I don't know. It's certainly weird and I suspect we changed something at some point, on one or both ends. It's been a long time since I wrote the Publication Service and being a pet project, it's perhaps not tested well enough - still, since we're talking about default settings here, it seems that this could have been found earlier if it had been like that forever. I don't know.

Second, is it possible to use SOAP encoding? Yes and no, oddly enough - another point where I'm not sure where a problem comes from. It is easy to switch the Publication Service to using SOAP, like this:

<Channels>
  <xpopub:HTTPTargetChannel>
    <Port>9999</Port>
    <UseBinaryEncoding>false</UseBinaryEncoding>
  </xpopub:HTTPTargetChannel>
</Channels>

The problem is, when I tried this, I got another weird exception coming up, saying that the SOAP formatter isn't able to deal with generic classes being serialized. Okay then... hm. The point where we changed XPO to have generic classes involved in the inheritance hierarchy is a really long time ago, so it's probably been like that since then. Interestingly, while I don't know how many people use Remoting with HTTP/SOAP, I'm pretty sure there are some people using XML Web Services, which use HTTP/SOAP as well. The answer to that is that the SOAP formatter for Remoting isn't the same as the one for XML Web Services. And they are both different from the one being used for WCF, if I'm not completely mistaken. Trust Microsoft to find ways of making complex things more confusing.

Summary of all the odd stuff above

If you've stopped reading my ramblings a while ago, here's the summary of important points:

The reason for the issue is that the XPO Publication Service and the Remoting client code in the XPO libraries use different formatter defaults for the HTTP channel. We should look into making some changes so this would be compatible automatically.
Using TCP instead of HTTP makes things work. This is probably the best idea for many cases, unless HTTP is absolutely required.
Using binary formatting with HTTP also makes things work. An additional line of code is required for this purpose, but inserting this into application initialization code shouldn't be a problem in most cases.
Using SOAP formatting with the Publication Service is easy, but it doesn't actually work for technical reasons in the Remoting SOAP formatter. We might look into this in a bit more detail to see if there's anything we can do about this.
Using the HTTP/SOAP combination is perfectly possible with XML Web Services as well as WCF, to my knowledge. XML Web Services are not self-hosting, so they can't really be supported by the XPO Publication Service. WCF can be self-hosting and it would be possible to extend the Publication Service to support it directly. The Publication Service can be extended externally, i.e. without changes to its own code base. With or without support in the Publication Service, there are plenty of articles on this blog that show how to tunnel through Web Services and WCF.

Update:

I've talked to the XPO team, and they have tracked down the problem with the SOAP formatter. Turns out this should actually work - so we regard it a bug and it will be fixed. Hang on... just now an IM tells me that the fix has been checked in internally. Will be in the next minor release.

The thing with the default behavior of the XPO Remoting code is that it uses defaults the .NET Framework provides, which is a good thing of course. I guess it would make sense to change the default for the Publication Service to use SOAP instead of binary... I'll probably do that soon.

ANN: DXCore XPO plugin 1.2.0.1

Oliver Sturm (Developer Express) — Thu, 07 Aug 2008 13:34:05 GMT

Version 1.2.0.1 of the XPO plugin is now available, compiled against DXCore 3.0.8. Please download it here:

CR_XPOFieldSync-1.2.0.1.zip (20489 bytes)

As always, if you're not familiar with the purpose of the XPO plugin, please read this description of the "Simplified Criteria Syntax" feature.

I have recently mentioned to a few of you, through the Support Center, that I was considering retiring the plugin soon. LINQ has been mentioned several times as the query technology of choice, and I would like to make that point again. The "Simplified Criteria Syntax" feature in XPO was originally introduced as a way to write queries more easily, and specifically with the fact in mind that LINQ was at that time still about two years away from general availability. Now that LINQ is available and well supported in XPO, we do recommend you use it to write queries in code. Since LINQ is a very general-purpose technology, there will be cases where our own criteria system can do things that can't be represented in a LINQ expression. But for the vast majority of cases, there's no better way of writing queries in code today.

Nevertheless I've changed my mind about deprecating the XPO plugin. The reason is that I've learned how many other things, apart from querying, users do with the plugin and the code it creates. Generally speaking, having a reference to a certain field in a persistent class - which is what the XPO plugin creates - is very useful in a variety of cases that don't have to do directly with querying, and so cannot be covered by LINQ. So for now, I will keep supporting the XPO plugin as well as time allows.

For version 1.2.x, I have restructured certain things about the way the plugin works. In prior versions, the plugin was trying to be rather clever about the code generation/modification process, for instance by removing only those elements that weren't needed anymore after a code change, or inserting only those that were new. This has proved extremely tricky over time with regard to the interaction with the DXCore and the code parsing it does, since the plugin needs to know exactly what the current state of the code is while it executes several deletions and insertions, and the developer possibly keeps on typing and making more changes. The major structural change in this version is that the plugin always creates the entire code block and either inserts it or replaces the previous version with it. This does come with its own problems - see below -, but it makes things much more predictable and less dependent on details of the inner workings of the DXCore.

Here's a summary list of changes in this version:

Restructured lots of the plugin to solve some long-standing problems
Manual execution of sync is now possible - configure a key binding for the SyncXPOClass action
Fixed B95475 - (a part of the) Fields class wasn't being created
Fixed B93229 - certain persistent property types weren't recognized correctly
Fixed a problem where the code region was created in the wrong place, or not at all, when using nested persistent classes
Fixed CB56295 - handling of NonPersistent on base classes
Implemented CS49600 - optional inclusion of NonPersistent members - see option dialog
Implemented AS13477 - toggle automatic sync using a key binding - configure a key binding to the ToggleXPOAutoSync action
The "Global Off" option has been replaced by the "Automatic Sync" option, which is Off by default

Note the last entry, please. If you like to use automatic synchronization, you have to go to the options dialog after installation of the latest version and switch it on. By default, the plugin doesn't use automatic sync anymore. From my own experience during testing, the manual sync is quite comfortable, although of course there's a risk of forgetting to sync.

There are currently three known problems with this plugin version:

The forever outstanding issue CB8627. Unfortunately, I still don't have a solution to this. The DXCore parser is apparently unable to tell me if there's a critical structural problem in the source code I'm looking at, and so the sync process creates code in the wrong place, or doesn't recognize that there's already a region it should replace. I'm still hoping to be able to fix this problem eventually. In this version, one possible workaround is to use manual sync - just make sure your project is valid (hit Shift-Ctrl-B to build, for instance) and if that's all fine, trigger a manual sync.
Due to some Visual Studio implementation oddities, (a) the newly created code region is always expanded when it is inserted and (b) it happens that it doesn't collapse automatically. There's code in place that tries its best to collapse the region and most of the time it works, but when you're tying furiously while sync kicks in - especially when you're hitting Return as well to create new lines - the region might stay open. This doesn't have any technical implications whatsoever, it just doesn't look nice. I'm afraid there will probably not be a solution to this.
Finally, I've seen the need to reintroduce the snag of the dead redo list in Visual Studio, as previously reported in B91445 and CB54533. Okay okay, I'm joking, I didn't do it on purpose. But I had to fiddle around with the undo handling after making those structural changes explained above, and for some reason everything now works differently... difference between using immediate and queued edits in DXCore, in case you're curious. I have now managed to make undo work really nicely - it simply skips the automatic changes and undoes (is that a word?) whatever you actually did right before - but Visual Studio still kills the redo list... opening and then discarding an undo unit apparently does that, and I doubt there's a way around it. My apologies to everybody who loves redo - personally I hardly ever use it, so I hope it's not going to be the worst kind of showstopper. Oh yeah, and I should mention that this problem applies only to automatic sync! So using manual sync is once more a workaround if you really want redo functionality.

So that's it - please try the plugin and let me know if you find any problems. I apologize in advance for all issues I've introduced now - when making rather major changes like I did, breaking something also becomes more likely.

XPO Publication Service update for 8.1.4

Oliver Sturm (Developer Express) — Sat, 31 May 2008 15:10:34 GMT

There are no changes in this version -- surprising as it may be, nobody ever reported a single bug in this tool. I expect that is because nobody is using it :-) Oh well.

If you don't have a clue what this is about, see these previous posts for details:

I updated the tool for XPO 8.1.4 and everything still seems to be working just like it did. Nice. Here are the downloads:

XPOPublicationService-1.0.1.0.zip (36849 bytes)

XPOPublicationService-1.0.1.0-bin.zip (40113 bytes)

ANN: DXCore XPO plugin 1.1.0.7

Oliver Sturm (Developer Express) — Fri, 11 Apr 2008 19:08:04 GMT

Once again, a new version of the plugin, with a few bug fixes. If you're not familiar with the purpose of the XPO plugin, please read this description of the "Simplified Criteria Syntax" feature.

This is the download for version 1.1.0.7: CR_XPOFieldSync-1.1.0.7.zip (21071 bytes)

Version 1.1.0.7 is compiled against DXCore 3.0.5. Here are the fixes I made:

Fixed B91445 and CB54533 - redo list being cleared by XPO plugins sync process.
Fixed Q99903 - no Field class entries for collection properties
Fixed part of B93229 - no Fields class entries for properties made persistent with ValueConverterAttribute

The issue of CB8627 is still outstanding, as described previously. I do have hope though that I'll be able to solve that problem soon - I was actually working on it, but then it turned out that there's a bug in DXCore that prevents it from happening at this time. I'll get back to that issue as soon as possible.

As usual, if you have any other problems with the plugin, please let us know!

XPO, LINQ and Distinct

Oliver Sturm (Developer Express) — Fri, 28 Mar 2008 17:12:00 GMT

XPO, LINQ and Distinct

In a recent forum post, Dusan Pupis mentioned that our LINQ support didn't currently include the Distinct extension method. I looked into this and I found a pretty complex situation that requires some explanation.

First, in XPO 8.1.1, we do in fact support Distinct(), i.e. the overload that doesn't take any further parameters, in some situations. For my test application, I'm using the following code based on three standard persistent classes:

XpoDefault.DataLayer = new SimpleDataLayer(
  new DataStoreLogger(
    new InMemoryDataStore(new DataSet( ), AutoCreateOption.SchemaOnly),
    Console.Out));

using (UnitOfWork uow = new UnitOfWork( )) {
  Artist eltonJohn = new Artist(uow) { Name = "Elton John" };
  eltonJohn.Albums.Add(new Album(uow) { Name = "Tumbleweed Connection", Year = 1970 });
  eltonJohn.Albums.Add(new Album(uow) { Name = "Don't Shoot Me I'm Only the Piano Player", Year = 1973 });
  eltonJohn.Albums.Add(new Album(uow) { Name = "Too Low for Zero", Year = 1983 });
  eltonJohn.Albums.Add(new Album(uow) { Name = "Captain Fantastic and the Brown Dirt Cowboy", Year = 1975 });
  eltonJohn.Albums.Add(new Album(uow) { Name = "Caribou", Year = 1974 });

  Artist rollingStones = new Artist(uow) { Name = "The Rolling Stones" };
  rollingStones.Albums.Add(new Album(uow) { Name = "Aftermath", Year = 1966 });
  rollingStones.Albums.Add(new Album(uow) { Name = "Their Satanic Majesties' Request", Year = 1967 });
  rollingStones.Albums.Add(new Album(uow) { Name = "Sticky Fingers", Year = 1971 });
  rollingStones.Albums.Add(new Album(uow) { Name = "It's Only Rock 'n' Roll", Year = 1974 });
  rollingStones.Albums.Add(new Album(uow) { Name = "Exile on Main Street", Year = 1972 });
  rollingStones.Albums.Add(new Album(uow) { Name = "Sucking in the Seventies", Year = 1981 });

  Artist procolHarum = new Artist(uow) { Name = "Procol Harum" };
  procolHarum.Albums.Add(new Album(uow) { Name = "Procol Harum", Year = 1967 });

  Artist jethroTull = new Artist(uow) { Name = "Jethro Tull" };
  jethroTull.Albums.Add(new Album(uow) { Name = "Thick as a Brick", Year = 1972 });
  jethroTull.Albums.Add(new Album(uow) { Name = "Minstrel in the Gallery", Year = 1975 });

  new TestData(uow) { Val1 = "One", Val2 = "Two" }.Save( );
  new TestData(uow) { Val1 = "Three", Val2 = "Four" }.Save( );
  new TestData(uow) { Val1 = "Four", Val2 = "Two" }.Save( );
  new TestData(uow) { Val1 = "One", Val2 = "Two" }.Save( );

  uow.CommitChanges( );
}

With this initialization, I can do a few things with Distinct. For example, I can find a distinct Album year like this:

using (UnitOfWork uow = new UnitOfWork( )) {
  var albumYears =
    from album in new XPQuery(uow)
    select album.Year;

  ObjectDumper.Write(albumYears.Distinct( ));
}

Since I'm using the logger in my data layer, I can see that the query is this:

SelectData request with 1 queries:select('N0'.'Year') from('Album'.'N0') 
  where(IsNull('N0'.'GCRecord')) order() group('N0'.'Year') having() params() ;

Along the same lines, I can also query more than one value, this time using the TestData objects:

var testdata =
  from test in new XPQuery(uow)
  select new { Val1 = test.Val1, Val2 = test.Val2 };

ObjectDumper.Write(testdata.Distinct());

This creates the following query:

SelectData request with 1 queries:select('N0'.'Val1','N0'.'Val2') from('TestData'.'N0') 
  where(IsNull('N0'.'GCRecord')) order() group('N0'.'Val1','N0'.'Val2') having() params() ;

As you can see, in both cases the XPQuery<T> has translated the query code into our query language, which in turn gets executed on the server. This is really the whole point of LINQ to XPO - bridging from LINQ syntax and extensions into the XPO world, where our database independent querying system can then execute the query efficiently on the server.

Now here's something that doesn't work:

var result =
  from artist in query
  select artist;

ObjectDumper.Write(result.Distinct());

Why not? Because the system can't guess what it is that makes one Artist different from another. Of course there's the primary key, but I don't need Distinct for that one. All other distinctions are much more complex to describe, so Microsoft introduced another overload of the Distinct method:

public static IQueryable Distinct(
  this IQueryable source, IEqualityComparer comparer);

This one takes a parameter for an IEqualityComparer implementation, and of course that interface allows me to specify precisely when a certain object is to be regarded equal to another object. However, this overload is not support by XPO, and will likely never be supported.

There are two reasons for that. First, Microsoft decided to have this method take an interface as a parameter, not a lambda expression, like most of the other LINQ extension methods do. Of course we could create an overload ourselves that takes a lambda expression, but that brings us to reason number two: the code that determines equality is likely to be so complex that translating it into something that would run on the server would be impossible in the majority of cases. I don't know if reason two is perhaps also the source of reason one, but it seems likely.

There's one easy way to use this overload of Distinct with XPO: simply make sure to use Enumerable.Distinct, pass in your IEqualityComparer and suddenly everything works. How does it work? Very simple: the data is retrieved from the server and Distinct runs over the client-side results, where the comparer code can be executed. We could have made this happen by default in the cases where translation isn't possible, but we decided it would be safer not to do that. So it's left to you to be explicit about this if your intention is to get Distinct executed client-side. Here's how that works (for example - there are many different ways of making IEqualityComparer work for you):

// This is bad!!! Just an example. This results in Elton John
// and Procol Harum being equals.
class Comparer : EqualityComparer {
  public override bool Equals(Artist x, Artist y) {
    return GetHashCode(x) == GetHashCode(y);
  }

  public override int GetHashCode(Artist obj) {
    if (obj.Name == "Elton John" || obj.Name == "Procol Harum")
      return 734;
    return obj.GetHashCode( );
  }
}

...

  var result =
    from artist in new XPQuery(uow)
    select artist;

  ObjectDumper.Write(((IEnumerable) result).Distinct(new Comparer( )));

The cast to IEnumerable<Artist> makes sure that the Distinct extension method is taken from the Enumerable class (because it's declared as an extension on IEnumerable<T>). Of course there are other ways - you could call ToList() on the result before calling Distinct, or call Enumerable.Distinct without using the extension method syntax.

Four new Knowledge Base articles on eXpress Persistent Objects

Nick (Developer Express) — Tue, 18 Mar 2008 19:07:00 GMT

During the last few weeks I've had some time to kill so I decided to fill up our Knowledge Base with new material on eXpress Persistent Object (XPO). For me, this product can be thought of like an iceberg: At its surface, one sees only a small portion of what's hiding underneath. With XPO, you need to know very little about the product to create your first application. But by digging a bit deeper you'll uncover many new aspects of the product and hopefully appreciate its strengths and power, and feel even more confident to place it as the foundation of your data-aware applications.

So, enough with the metaphors...here are a few articles to help you dig deeper into XPO in many unique ways:

How to use XPO in an ASP.NET (Web) application

Getting started with XPO development for Compact devices

How to create a criterion, which returns an intersection of two many-to-many sets

How to implement self-referenced persistent objects, and to display these objects within the XtraTreeList

Some SetPropertyValue/GetPropertyValue overloads will be deprecated

Oliver Sturm (Developer Express) — Fri, 14 Dec 2007 15:22:00 GMT

In future releases of XPO, we will deprecate certain overloads of the SetPropertyValue and GetPropertyValues in XPBaseObject. In a previous post of mine I have outlined some of the various combinations that can currently be used for property implementations in persistent classes, and the case that's the subject of this post is case 1.

There's a nice little example and an explanation of this case in the original post, but here's a quick summary. The overload in question is the one where the methods do not take a parameter for the property name. This was previously already a scenario that I marked not recommended, because of performance as well as technical issues with it. The performance issues are simply due to the additional work that needs to be done on every single property access, the technical issues I'm referring to relate to changes in .NET 2. Again, for more details about the issues we're having with this, please read this blog post or feel free to contact our support team and request any information you need.

We have since had a number of support incidents, where programmers were not aware of the implications of this particular implementation, and were wondering about various side effects. In each of those cases, the problem could be solved simply by using one of the method overloads that take the property name as a parameter. As things are, we regret that we ever introduced this seemingly comfortable way of working with the helper methods. As there are now no perceivable advantages left of this approach, we have made the following decision:

Over the next few releases, these method overloads are going to be marked obsolete, and then removed!
(Clarification: we are still talking about those overloads of SetPropertyValue and GetPropertyValue that do not take a property name as one of their parameters!)

As a result, if you are using these overloads, you will start seeing warning messages from the Obsolete attribute in one of the next XPO versions (I'm currently assuming 7.3.6, as 7.3.5 is just out the door). To solve this problem and make sure that your application does what it should (isn't it nice that at least there's some real purpose in this <g>), you will have to insert the property name into the parameter lists of all calls to SetPropertyValue and GetPropertyValue (it's always the first parameter).

Please feel free to contact us in any way if you want to discuss the matter with us, register disagreement, or if you have questions.

XPO - Using the class generation wizard

Oliver Sturm (Developer Express) — Tue, 24 Jul 2007 15:11:37 GMT

Newest versions of XPO (since the 7.2 release) have a wizard that allows you to to create class structures from existing databases. This feature has been requested many times in the past, and we made first steps a while ago by including a tutorial project that supports MS SQL Server (if you're curious, the tutorial is still included - it's called "GeneratingPersistentClasses"). The new wizard takes things to the next level by supporting a variety of additional database systems, so we recommend you start using it instead of the tutorial application. If you give us feedback on the functionality, we will also consider extending it in the future.

Using the feature is not difficult, but possibly not immediately obvious. In an existing project (currently with the exception of ASP.NET projects - see the end of this post), add a new item to the project and select Persistent Classes X.X from the list of item types (the X.X stands for the precise version of XPO you have installed). You can change the name of the new file to something more useful, but in practice I don't find that very important because I'm going to change the file name anyway once the classes have been generated.

Before anything is created, the template brings up a dialog that allows you to select the database provider you want to use, the database itself and any connection parameters, and finally the tables and columns you want to generate classes and properties for.

After this step, the class creation process is executed and the result is a new file in your project with all the new classes inside. The wizard currently leaves two important things to do:

Renaming the classes. In traditional databases, tables regularly have weird names that I don't want in my OO class structure. Even if the names aren't weird, they may not be totally suitable. In the screenshot above you can see tables called Categories and Customers (plural). I would rename the resulting classes to Category and Customer (singular), because that makes more sense in an OO context.
Hint: if renaming sounds like a lot of work, have a look at Refactor! It has a Rename refactoring that makes this extremely easy.
Pull each of the new classes into its own file. Having a single class per file is a convention I recommend very much. Another thing Refactor! can help you with, btw. Why is it not done automatically? No idea, to be honest... but I think the current implementation isn't so bad, because it leaves the decision to the developer and it makes search and replace type operations on the freshly generated source code much easier (for those who don't use a tool like Refactor! <g>).

Finally, a word about ASP.NET projects. In the current release, the template Persistent Classes X.X (as well as the standard template Persistent Object X.X) is not available if the project type is ASP.NET. The reasons for this are somewhat complicated, you can find an explanation here. We will fix this problem for the next minor update, meanwhile you can import persistent class declarations into a different project and move them over to the ASP.NET one - but do look out for the guidelines given in this post.

XPO and ASP.NET - Where to put your persistent classes

Oliver Sturm (Developer Express) — Tue, 24 Jul 2007 15:00:54 GMT

If you have been working with XPO in ASP.NET applications, you may have noticed that the standard XPO Persistent Object X.X (and also the new Persistent Classes X.X) template is not available when the current project is an ASP.NET project. We recently re-evaluated the decision to leave the templates out from that location, and while we found the original reason to be invalid by now, there are still some things to be aware of when working with persistent classes in ASP.NET projects.

Back in the days of beta testing for VS 2005, we first discovered a problem that appeared when persistent class hierarchies were used in ASP.NET projects. The reason for this issue was that XPO stores certain information about classes inside the database, among those the name of the assembly from which the class was originally read. A lot of the time this information is not used, but when persistent classes are derived from other persistent classes and instances of those derived classes are created and saved, XPO looks up the assembly name information and tries to demand-load the correct assembly.

In ASP.NET 2.0/VS 2005, a project is built (by default) into several assemblies, and the names of these assemblies include a "key" part that changes regularly after rebuilding. So when XPO stores away the assembly name of a certain class in a first application run, then that information is likely going to be invalid on a later run because by now the name of the assembly has changed.

When we discovered this issue originally, we found two workarounds. The first workaround is to enable a project level option in VS 2005, called Use fixed naming and single page assemblies (you can find it on the MSBuild Options page). While this makes sure that the assemblies have reliable static names, it has the major disadvantage of creating lots of little assemblies, because every single (web) page now gets its own. Well.

The second workaround is still our current recommendation, which is to use a separate assembly for your persistent classes -- a standard "class library" project type will do for this. Not only did this solve the issue above, it is also generally a good practice because it allows you to easily create other applications that use the same classes as your web application. Think of admin frontends running on Windows Forms, database update utilities and so on...

Now, in our recent re-evaluation we found something that was news to us: there is special magic implemented in the .NET framework, so that assemblies with a name that starts with App_Code will be found even if the rest of their name doesn't equal the search string. In other words, a call to Assembly.Load("App_Code.myRandomPostfix") will load the actual App_Code assembly when used from a web application, never mind the fact that the actual assembly obviously uses a different postfix. This magic handling seems to be in place for .NET 2.0 and above. We're not quite sure why we didn't see it earlier, but probably it was added before the final 2.0 release.

This discovery means that technically your persistent classes are safe when stored inside the App_Code subfolder of your ASP.NET project. We will therefore make the templates available for use in those projects in the next minor update. I would still recommend you use a separate assembly, but at least we can get rid of a somewhat arbitrary restriction now.

XPO and ASP.NET - a look at the initialization code

Oliver Sturm (Developer Express) — Fri, 20 Jul 2007 14:26:00 GMT

Mehul got back to me yesterday and asked me to explain the XPO related initialization code that is being used in our new ASPxGridView demo. This is the code in question:

    void Application_Start(object sender, EventArgs e) 
    {
        if(DemoSettings.IsSiteMode) {
            ConnectionStringSettings conn = 
                ConfigurationManager.ConnectionStrings["XPOServerMode"];
            if(conn != null) {
                // Code that runs on application startup
                DevExpress.Xpo.Metadata.XPDictionary dict = 
                    new DevExpress.Xpo.Metadata.ReflectionDictionary();
                dict.GetDataStoreSchema(typeof(ServerSideGridTest).Assembly);
                DevExpress.Xpo.DB.IDataStore store = 
                    DevExpress.Xpo.XpoDefault.GetConnectionProvider(
                    conn.ConnectionString, 
                    DevExpress.Xpo.DB.AutoCreateOption.SchemaAlreadyExists);
                store = new DevExpress.Xpo.DB.DataCacheNode(
                    new DevExpress.Xpo.DB.DataCacheRoot(store));
                object layer = new DevExpress.Xpo.ThreadSafeDataLayer(dict, store);
                Application.Add("XpoLayer", layer);
            }
        }
    }

Great. Now what's happening here? Let's go over the method line by line:

    if(DemoSettings.IsSiteMode) { 
        ConnectionStringSettings conn = 
            ConfigurationManager.ConnectionStrings["XPOServerMode"];

These are demo specific lines, nothing to do with XPO. In fact, when working with XPO it is a common practice to use helper methods on certain XPO classes to retrieve an auto-generated connection string instead of using a pre-created one. For SQL Server, for example, you'd want a call into DevExpress.Xpo.DB.MSSqlConnectionProvider.GetConnectionString( ... ); to get a connection string. There's nothing wrong with the approach of storing away the connection string, but it's certainly a little easier to break.

    DevExpress.Xpo.Metadata.XPDictionary dict = 
        new DevExpress.Xpo.Metadata.ReflectionDictionary();

The dictionary of metadata that XPO works with contains important information about the persistent classes in your application, and it is associated with the so-called data layer (an object that implements IDataLayer, if you want to be technical about it). In general, a data layer is only created once per application, so the metadata dictionary is also created just once.

In a Windows Forms application the code related to configuring the data layer is typically much simpler, probably something like this:

    XpoDefault.DataLayer = XpoDefault.GetDataLayer( ... );

In ASP.NET, things are not quite as simple because we have to deal with the fact that we're working in a multi-threaded environment. If you take a little peak ahead in the complete code of the Application_Start method above, you will see a class called ThreadSafeDataLayer being instantiated. This specific implementation of IDataLayer can work in multi-threaded applications, but it requires a metadata dictionary that is stable during the application's lifetime. A data layer in a Windows Forms (single threaded) application would be extended during the application run, depending on the classes that are being used. This is not allowed to happen in the ASP.NET multi-threaded scenario, and that's why we have to create and fill the metadata dictionary manually, and before creating the data layer.

    dict.GetDataStoreSchema(typeof(ServerSideGridTest).Assembly);

This line does the work: it needs an assembly reference (or several actually, it is possible to pass in more than one assembly reference at once), finds all the metadata in that assembly and adds it to the dictionary. To get to the correct assembly reference, one of the persistent types in the application is used to find the reference dynamically. This is an arbitrary persistent type in your application -- it looks a bit a hack, but it's one of the easiest ways of getting hold of such a reference.

    DevExpress.Xpo.DB.IDataStore store = 
        DevExpress.Xpo.XpoDefault.GetConnectionProvider( 
        conn.ConnectionString, 
        DevExpress.Xpo.DB.AutoCreateOption.SchemaAlreadyExists);

Now we are ready to create a connection provider object, which is an object that implements the interface IDataStore. This is basically the DB specific driver that XPO needs to communicate with your backend of choice. The XpoDefault.GetConnectionProvider( ... ) method analyzes the connection string to find out which type of connection provider it needs to create. For connection strings that are retrieved from the XPO helper methods (see above), there's a special flag inserted into the connection string that simplifies identification. In the ASPxGridView demo, the connection string is read from the Web.config file, but it still contains the flag. Here's the string (wrapped):

    XpoProvider=MSSqlServer;data source=(local);user id=servermodeprojects;
        password=smp$;initial catalog=ServerModeGridProjects;Persist Security Info=true

The first parameter XpoProvider tells the GetConnectionString method that it needs to instantiate the connection provider for MS SQL Server in this case. If your connection string doesn't have such a flag, there are heuristics in that method trying to recognize the string nevertheless, but this may not work totally reliably. You should either retrieve the connection string from one of the helper methods or make sure the flag is in there if you store the complete connection string in a config file.

Finally, there's an enum AutoCreateOption being passed into that method. This tells XPO how to handle the database schema - SchemaAlreadyExists says that the schema shouldn't be created, because it is expected to be there already. There are several other alternatives documented in the XPO product documentation.

    store = new DevExpress.Xpo.DB.DataCacheNode( 
        new DevExpress.Xpo.DB.DataCacheRoot(store));

This line wraps the IDataStore in instances of DataCacheRoot and DataCacheNode. DataCacheNode implements IDataStore itself, so the result of the operation can be used anywhere a "normal" connection provider can also be used. The purpose of the wrapping is to establish data layer caching, as described in this technical article.

    object layer = new DevExpress.Xpo.ThreadSafeDataLayer(dict, store);

With the metadata dictionary and the connection provider in hand, we can finally create our thread-safe data layer. I don't know why the person who wrote the demo used object as the type -- it should really be something explicit, must likely IDataLayer.

    Application.Add("XpoLayer", layer);

This line of code stores the data layer away for later use. Apart from the ASP.NET specific approach used here, there's also a mechanim built into XPO to set a default data layer: the static XpoDefault.DataLayer property. The main difference is that if XpoDefault.DataLayer has been set, it is no longer necessary to pass in an explicit data layer reference when constructing instances of Session or UnitOfWork. Of course there's the general argument that this works only as long as just one data layer is being used in an application, but that is really very common -- as I hear, the approach was chosen in this case becauses use of the Application class for this kind of thing is regarded as a good practice in ASP.NET applications. Wouldn't necessarily convince me, but there you go.

Receiving SqlDependency notifications for use outside XPO

Oliver Sturm (Developer Express) — Fri, 25 May 2007 12:56:00 GMT

Well, there's obviously one really easy way to do it: just hook into the SqlDependency feature independently from XPO and receive the notifications that way. But there's an easier way to do it if XPO is hooking these notifications anyway. I got some code from an XPO developer and I created a sample around it to show how it works.

Note: Before that developer agreed to give me his code, he made me promise to say this: we really don't recommend you use this! It is an approach that will have severe performance drawbacks in many multi-user scenarios. If you end up actually using it, please make sure you do testing on your application in circumstances that resemble production use. If you find that you have performance problems - which we believe is very likely -, we will not be able to do anything about it on the XPO level. Please consider yourself warned.

Now you're probably asking: "why is he going to post a sample if he doesn't want me to use it?" Well, I believe that it is better to explain things to people, including (potentially) bad things, and let them make up their own minds. So here we go.

The sample I'm using is the same I showed in the previous post about the SqlDependency feature. You can download the whole source code below. For the purpose of notification, the sample now includes a new class called DataStoreChangeTrackingWrapper, which has two important (and a bunch of not as important) methods:

	...
	protected void ProcessChanges(DataCacheResult result) {
		CurrentCookie = result.Cookie;
		if (result.UpdatedTableAges == null)
			OnEverythingChanged( );
		else if (result.UpdatedTableAges.Count > 0)
			OnTablesChanged(result.UpdatedTableAges.Keys);
	}
	...
	public void CatchUp( ) {
		DataCacheResult result = Root.NotifyDirtyTables(CurrentCookie);
		ProcessChanges(result);
	}
	...

The DataStoreChangeTrackingWrapper class is a data store implementation. It overrides the ProcessUpdateSchema, ProcessSelectData and ProcessModifyData methods and makes sure that ProcessChange is called during the execution of all these methods. So if the application accesses the data store regularly (like the sample application does in its loop), the OnEverythingChanged and OnTablesChanged methods are going to be called automatically. The CatchUp method is for those cases where the regularity is not there - a timer could be used to call into CatchUp regularly to make sure that updates are passed through.

The two methods OnEverythingChanged and OnTablesChanged are those that get called once changes have been detected in the database. OnTablesChanged is the more common of the two, and it gets more detailed information as well. OnEverythingChanged is called only in cases where the cache gets disconnected - for instance when the cache root service is stopped in the case of a remotely published cache root.

If you run the sample application and follow the same steps as in the previous article, you should see that at the point where external changes are being discovered, information about them will appear in the console. The sample uses a 5 second sleep in the main loop (previously one second), so that it's easy to spot the additional output line.

Download the sample here: XPOSqlServer2005CacheAndNotification2.zip (4640 bytes)