XPO is good for distributed applications

XPO Team Blog
13 April 2006

XPO 6.1 introduces a number of changes that do a lot for its support of distributed applications. This is going to be the first in a series of posts around the topic of distributed applications involving XPO. I hope I can shed some light on the structures we provide in XPO for this purpose and the scenarios in which various platforms like .NET Remoting, XML Web Services and Windows Communication Foundation (WCF) can be used in conjunction with XPO.

As always, please feel free to comment on these posts as much as possible – at some point, I might summarize all the information in a more formal technical article, and it would be good to have as many different perspectives in there as possible.

So why does XPO work together well with distributed systems in general? The answer to this question requires some knowledge about the architectural requirements of a performant setup of such systems, as well as about the new layered structure of XPO’s inner workings.

Making distributed systems perform well

To make distributed systems perform well, we should make sure that as little data as possible is transferred over the wire. (Depending on his/her definition of “performing well”, the programmer might also give some thought to topics like platform and/or protocol independence, but that’s not what I’m talking about here.) To make as little data go over the wire as possible, it is important to define those interfaces that are called remotely very efficiently. This means two things:

  1. The data that is being transferred must be structured specifically for this purpose.
  2. The structure of the application, and the interface itself, must be designed in such a way that calls to the remote interfaces are minimized.

The result of this will be an interface that’s coarse-grained and that works with data types different from those that are being used on either side of the communication.

So what does XPO do in this regard?

XPO defines two different interfaces that are structured in the way described above: IDataStore and ICacheToCacheCommunicationCore.

Xpo-distributed-layers

This is how IDataStore is defined:

public interface IDataStore {
  AutoCreateOption AutoCreateOption { get; }
  ModificationResult ModifyData(params ModificationStatement[] dmlStatements);
  SelectedData SelectData(params SelectStatement[] selects);
  UpdateSchemaResult UpdateSchema(bool dontCreateIfFirstTableNotExist, params DBTable[] tables);
}

As you can see, the data types that are being passed to and returned by these methods are generally (with the exception of the AutoCreateOption enum and the bool primitive data type) not the same that you are used to handling in your own applications, i.e. business objects. They are also not the same that are being handled by your database backend, for example ADO.NET structures or even SQL code. These are data types strictly designed for the purpose of being passed through the IDataStore interface, of course with the idea in mind that the interface may be implemented remotely.

ICacheToCacheCommunicationCore is only minimally bigger than IDataStore:

  public interface ICacheToCacheCommunicationCore {
    DataCacheModificationResult ModifyData(DataCacheCookie cookie, ModificationStatement[] dmlStatements);
    DataCacheResult NotifyDirtyTables(DataCacheCookie cookie, params string[] dirtyTablesNames);
    DataCacheResult ProcessCookie(DataCacheCookie cookie);
    DataCacheSelectDataResult SelectData(DataCacheCookie cookie, SelectStatement[] selects);
    DataCacheUpdateSchemaResult UpdateSchema(DataCacheCookie cookie, DBTable[] tables, bool dontCreateIfFirstTableNotExist);
  }

These two interfaces make it possible to cross boundaries to other systems in two places in the layer structure, depending on whether you want to use caching or not.

Free DevExpress Products – Get Your Copy Today

The following free DevExpress product offers remain available. Should you have any questions about the free offers below, please submit a ticket via the DevExpress Support Center at your convenience. We’ll be happy to follow-up.
Anonymous
Ryan Britton
Great Article Oliver - especially as a follow up to your article at the end of March.  I have been through the CacheRoot and CacheNode examples that ship with XPO and tried to consume as much of the documentation as I could... I must say that there is a marked shortage of code commentary and support material in aiding the understanding of this topic. I guess because it is at the forefront of XPO's advancement as a data persistence framework.

In terms of the management of distributed applications in an online/offline scenario where there is a local data store that needs to be synchronised with a central database - How would you approach this within the XPO architecture?

I'm really interested in the specific application of XPO in distributed scenarios. When do you anticipate better examples and use-cases being available for consumption and the expansion of the documentation to cover these types of scenarios? is there some other support material that I can consume in the interim?
13 April 2006
Anonymous
XPO
In my recent post XPO is good for distributed applications I explained the basics of support for distributed...
20 April 2006
Anonymous
XPO
First of all, my apologies – I haven’t had time lately to continue the series of posts about...
5 May 2006
Anonymous
Carel Lotz
Oliver

You mention the following:
> It's possible to build cache hierarchies out of a single DataCacheRoot instance and any number of DataCacheNode instances - this makes sense when certain parts of an application (or certain applications, thinking of client/server setups) need to use different settings for their data access, such as particularly current data.

Can you elaborate a bit more on the cache hierarchy idea?  It sounds promising - some code would be helpful. We have a database that contains readonly lookup style data and then instance data that users add via the application.  It is a multi-user application so we are using short living Sessions tied to a "logical unit of work".  This implies the local object cache of the Session is not going to be used too much.  However, we want to use DataLayer caching to cache the readonly data as to prevent hitting the database everytime for the lookups etc.

How do we go about setting up the DataLayer and the DataCacheRoot and DataCacheNode to support this?  We want to cache the readonly data, but not the instance data.  

Thanks
9 May 2006
Oliver Sturm (DevExpress)
Oliver Sturm (DevExpress)
Carel,
Please make sure to post your comments to the right blog post, otherwise it can get very confusing :-) Or in case you're confused now, it seems to me that your comment fits much better for the Session Management and Caching post.

Now, setting up a cache hierarchy is really very simple, and I do precisely that in the Session Management and Caching post, in the first code sample in the Data Layer Caching section. In that sample, there are two DataCacheNode instances being created for a single DataCacheRoot, and two separate data layers are created for them. To make things more complicated, you could introduce any number of DataCacheNodes, accessing each other in a hierarchical manner, as long as you make sure there's always a DataCacheRoot at the root of the whole construct.

What I don't do in the sample is a very important thing: I don't change anything about the settings of the nodes. So in this case, while I do have more than one DataCacheNode, it really doesn't make any difference, because they are configured to behave the same. In a real-world use case, you would now change parameters on one or all of the DataCacheNode instances, right now this means the MaxCacheLatency. So in your specific scenario, you would want to set the MaxCacheLatency on one of your DataCacheNodes to a very low value, so there's essentially no caching taking place (this is the node that you would use for access to the current data that you don't want to be cached). The second of your DataCacheNodes you would leave as is, or maybe you would even increase the MaxCacheLatency, and use that for access to your read-only data.

Please be aware that this approach must be applied very carefully. Because objects loaded from different cache nodes are necessarily also loaded from different sessions, they can't interoperate arbitrarily. So you have to be careful, if you handle objects from both sources in the same algorithm, exactly what you do with them.

And finally: are you sure you really need to do this at all? It's not really about caching the read-only data, that's what the data layer cache does in a default setup. This specific setup is about the conscious decision *not* to cache the other data - of course there are use cases out there where this is a necessity, I just want to make sure you have established that this is one of them.
11 May 2006
Anonymous
Carel Lotz
Oliver

Yes, this is actually the wrong thread - I was jumping around between these two threads and it got posted to the wrong one - you're welcome to move it to make it more cohesive and less coupled ;-)

Thanks, you answered my question.  I'll consider using the MaxCacheLatency to tweak the caching behaviour for the instance data whilst heeding your warning about actually deciding to not cache the data.  And yes, I need to make sure that I don't mix and match the objects created from the different DataCacheNodes as they will live in different Sessions.
11 May 2006
Anonymous
XPO
Geoff Davis commented on my announcement post: I just don’t get it! Not a problem, Geoff,...
16 October 2006
Anonymous
XPO
I just remembered that this was something that was still waiting to be published on the XPO blog. Here’s...
6 December 2006
Anonymous
XPO
And another “leftover” XPO sample – this time tunnelling through Windows Communication...
6 December 2006
Anonymous
Neri Cervin
Exists some practical example of the use of XPO with Remoting?
28 December 2006
Anonymous
XPO

SqlDependency is a SQL Server 2005 feature that allows the database server to notify a client when changes

16 May 2007
Anonymous
Alireza

Thanks for your good post.

There are Different ORM for .net, what are your selection criteria between them? for example between Dataobjects.net and XPO?

I want to implement a ORM remotely, and my desktop and web application can access it via internet, what can I do?

Best Regards

15 January 2008
Anonymous
XPO

And another “leftover” XPO sample – this time tunnelling through Windows Communication

28 March 2008
Anonymous
XPO

I just remembered that this was something that was still waiting to be published on the XPO blog. Here’s

28 March 2008
Anonymous
XPO

Geoff Davis commented on my announcement post: I just don’t get it! Not a problem, Geoff, and certainly

28 March 2008
Anonymous
XPO

First of all, my apologies – I haven’t had time lately to continue the series of posts about

28 March 2008
Anonymous
XPO

In my recent post XPO is good for distributed applications I explained the basics of support for distributed

28 March 2008
Dan Arnold
Dan Arnold

How do you handle errors for an intermittent internat connection. i.e. I make a call and it fails due to loss if connectivity. Do you have a recovery strategy that you can suggest, that would allow you to stay in the same application context and try the operation again.

Any insights in this area would be greatly appreciated.

20 May 2008
Oliver Sturm (DevExpress)
Oliver Sturm (DevExpress)

Hi Dan,

We have a mechanism in place that checks the connection before something important is done in our connection providers, and tries to reestablish it when it was lost. If an operation fails on the application level, you can be sure that reconnecting has already been tried - but you could still try to CommitChanges again if you think it makes sense.

There's an event on the SqlConnectionProvider class called OnReconnected, which is triggered when a connection has been established again. The purpose of this is to check the database to be sure it's the right version, and that kind of thing.

So that's the extent of what XPO currently does for you in this area. Going beyond this would mean having real offline client functionality, and we don't currently have that. One thing we regularly suggest when people ask about offline clients is to consider using SQL Server Compact Edition locally and set up synchronization with the "main" SQL Server instance somewhere else. In cases where connections are really flaky this could be a good step, though of course it comes with its own drawbacks (which are similar to other offline client solutions of course).

21 May 2008
Anonymous
eXpress App Framework Team

This is post no. 10 in the mini series "10 exciting things to know about XAF". You can find

31 May 2008
Dan Arnold
Dan Arnold

Thanks for very complete answer to my "loss of connectivity question". In fact, it was so good that I have another question for you.

I have not found a great way to get useful information back from the exceptions that might be thrown from the (remote)IDatastore interface.

How can I distinquish a concurrency error from a index violation?

Scenario:

Multiuser smartcleint application. I create a new customer named Bob, but in the meantime another rep has created the customer Bob. I get an exception, but its too course grained (SqlException).

I wish XPO threw more fine grained exceptions, like IndexViolationException (with field or index property), OptimisticLockFieldException ( or whatever )

Any thoughts on how you would handle scenarios like this?

20 July 2008
Anonymous
DbWorld ?? Blog Archive ?? [Dbworld] CFP: The 5th IEEE Workshop on … | Ontology Engineering Addict

Pingback from  DbWorld ?? Blog Archive ?? [Dbworld] CFP: The 5th IEEE Workshop on … | Ontology Engineering Addict

22 February 2010
Anonymous
DbWorld ?? Blog Archive ?? [Dbworld] CFP: The 5th IEEE Workshop on … | Ontology Engineering Addict

Pingback from  DbWorld ?? Blog Archive ?? [Dbworld] CFP: The 5th IEEE Workshop on … | Ontology Engineering Addict

22 February 2010
Anonymous
XPO

Introduction First, we would like to refresh your memory and return to the distant past and our old posts

17 May 2011

Please login or register to post comments.