Announcing the XPO Publication service

13 October 2006

Prompted by a customer’s request (thanks, Kenneth!), I have created a publication service for XPO data stores. I’m making the source code of the project available, so it may serve as a sample of a number of techniques.

The downloads

If you want to recompile the projects or simply have a look at the source code or the functional tests I wrote, you’ll need the source code archive: XPOPublicationService-1.0.0.0.zip

If all you need is the binary program (you can still add your own extensions, at least you should be able to), you can get the binary archive instead: XPOPublicationService-1.0.0.0-bin.zip

Requirements of the project

Starting from Kenneths original request, I worked with the following requirements:

  • The resulting program should be usable as a Windows service, to publish via .NET Remoting an XPO compatible data store (IDataStore).
  • It should be configurable (without recompiling) to work with various sources for this publication.
  • It should also support further extensions, such as defining Remoting channels to use, and chaining IDataStore providers, as required by the XPO data layer caching system or the data store logger.
  • A requirement I found myself during initial planning is that it must be extensible “externally”, so that another developer can create IDataStore implementations or Remoting channels in a separate assembly and use them in a publication, without having to change the source of the XPO Publication service.

So far I’ve left out the idea of publishing as a Web Service instead of via Remoting – I guess it might be possible to extend in that direction, if I find the time I’ll look into it.

So what's in the archives?

The source code archive contains a Visual Studio 2005 solution with four projects:

  • XPOPublication is the name of the functional implementation, which comes in the form of a class library, to make it easy to reuse in various contexts.
  • XPOPublication.Tests is an assembly of NUnit tests, which contains functional tests of various parts of the implementation.
  • Publish is a console application that lets you easily run publications from the command line.
  • XPOPublicationService is a Windows service wrapper for the publication functionality.

The binary archive includes the compiled versions of Publish.exe, XPOPublicationService.exe and XPOPublication.dll, as well as the configuration files for both executables and a sample publications.xml file (for details of configuration see below).

Configuration

Configuration of the publications is done via an XML file. The name of the file can be passed to the console application on the command line, otherwise a file called publications.xml in the current directory is used. For the Windows service, the name of the publication configuration file is taken from the application configuration file (XPOPublicationService.exe.config), and it is pre-configured to “..\..\publications.xml” for testing purposes.

The publication configuration file itself has roughly the following format (no, I haven’t created a schema for it… it’s very dynamic in large parts, so I didn’t really see the point in that):

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<XPOPublication>
  <Publication Name="MyPublication">
    <Source>
      < … source class definition … />
    </Source>
    <Target>
      < … target class definition … />
    </Target>
    <Links>
      < … link class definition 1 … />
      < … more link class definitions … />
    </Links>
  </Publication>

  … more publications …
</XPOPublication>

So what are all those “… class definition” things? Well, here I took a page out of XAML’s book. Pretty simple, really: the classes mentioned in these sections are instantiated directly from the XML code when the config file is read. That’s a good idea because it leaves us with the greatest flexibility – a class referenced in the config file needn’t be one that the tool developer (me!) ever heard of before.

The ConnectionStringSource

For example, there’s a predefined class called ConnectionStringSource, which uses a database connection string together with the XpoDefault.GetDataStore() method to create a connection provider for the publication. To use this class, the Source section might look something like this:

    <Source>
      <xpopub:ConnectionStringSource>
        <ConnectionString> … database connection string here … </ConnectionString>
        <AutoCreateOption>DatabaseAndSchema</AutoCreateOption>
      </xpopub:ConnectionStringSource>
    </Source>

In addition to this code, it is necessary to let the loader know where the class actually comes from. Note the "xpopub" prefix I'm using – this corresponds to an xmlns mapping that needs to be included on the XPOPublication node, like this:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<XPOPublication
  xmlns:xpopub="clr-namespace:XPOPublication">
  …

Again, this is the same syntax XAML uses for namespace and assembly references. The “assembly=” part of the mapping is also supported, if needed, so this allows you to reference classes in your configuration file that live in assemblies separate from my distribution of the tool.

As I recognize that the ConnectionStringSource will probably be the most used source, I have added attributes on the Publication element that allow to specify the two parameters of the class. So if there’s a SourceConnectionString attribute on the Publication class, the Source section (if there is one) will be ignored! The corresponding attribute for the AutoCreateOption is called SourceAutoCreateOption. To be clear, the above sample could be written like this instead:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<XPOPublication>
  <Publication Name="MyPublication"
    SourceConnectionString=" … database connection string here … "
    SourceAutoCreateOption="DatabaseAndSchema">
  … no source section! …

The RemotingTarget

The “standard” (probably most used) target is called RemotingTarget. A Target section utilizing the RemotingTarget class could look like this:

    <Target>
      <xpopub:RemotingTarget>
        <Channels>
          <xpopub:HTTPTargetChannel>
            <Port>2635</Port>
          </xpopub:HTTPTargetChannel>
        </Channels>
        <ServiceName>publication.rem</ServiceName>
      </xpopub:RemotingTarget>
      <RemotableClassCreation><xpopub:CacheRootRemotableClassCreation/></RemotableClassCreation>
    </Target>

As you can see, a Target definition uses the same system again to reference classes. The HTTPTargetChannel, of course, creates an HttpChannel for Remoting. There are also the TCPTargetChannel and the IPCTargetChannel, the latter with a property called “PortName” instead of “Port”.

The last line of the Target section defines the RemotableClassCreation strategy by using the CacheRootRemotableClassCreation implementation. This results in a CacheRoot being created, and the published object should then be accessed via the ICacheToCacheCommunicationCore interface instead of the simple IDataStore (this article about caching in XPO explains this in some more detail). The standard strategy for class creation is called DefaultRemotableClassCreation and it is automatically used if the RemotableClassCreation element is simply left out from the config file.

Finally, similar to the Source section above, there’s also a convenience attribute for the Target section. With the exception of the RemotableClassCreation element, the above configuration could more easily be written like this:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<XPOPublication>
  <Publication Name="MyPublication"
     Target="http://localhost:2635/publication.rem">
  … no target section! …

Also similar to above, if the Target attribute is there, a Target section will be ignored – so for advanced things like the RemotableClassCreation, or to set the EnsureSecurity property on the channels, the full syntax must be used. But the convenience system is extensible and it’s possible to register your own recognition algorithm…

The Links

Finally, there may be elements that need to be inserted into the chain of IDataStore implementations. These elements can be specified in the Links section of the configuration. There’s only one such element in the box, which is the FileLoggerLink. It includes a DataStoreLogger object into the chain, configured to log to a file. A Links section with a FileLoggerLink would look like this:

    <Links>
      <xpopub:FileLoggerLink>
        <FileName> … file name with path here … </FileName>
      </xpopub:FileLoggerLink>
    </Links>

When there’s more than one link defined, the order of links in the chain may become important. By default there’s no guarantee about the order in which the links would become part of the chain. But each chain link class should have a property that lets you define the order of links in the chain. For the FileLoggerLink this is called ChainIndex, which I’d like to propose as a standard if you’re going to create your own link classes. If you use this index value, it’s your own responsibility to make sure that you use the index values correctly – the library simply sorts the links by this index and instantiates them in that sorted order.

Troubleshooting

The library is instrumented in the most important places to provide Trace (System.Diagnostics) output. A trace switch is in use called XPOPublication, which you can configure in your application config file. The default application config files for Publish.exe and XPOPublicationService.exe already contain the necessary XML code, and the level is configured to 4 (everything) for Publish.exe and 2 (Warnings and Errors) for XPOPublicationService.exe. Likewise, Publish.exe is pre-configured to log to the console, and XPOPublicationService.exe logs to the system event log by default. Change these settings as needed to find problems.

I have purposefully handled exceptions sparingly in the library. In the console application, any exception messages are written out to the console, and in the Windows service they make it through to the event log.

It’s beta, even if it’s called 1.0

Let me conclude this post by saying that this software has not been tested extensively. I find this important to mention for two reasons:

  1. As always on this blog, I want your feedback. Please tell me what you think, what you don’t understand, that my code sucks, what suggestions you have.
  2. If you use the code in a real-world deployment and everything crashes around your ears, don’t look to me. I won’t feel responsible. Let me state this clearly: the code is untested in real-world scenarios and it has to be regarded as sample code only. You are yourself responsible for any problems, if you deploy it.
Tags
11 comment(s)
Lu. Po.
Lovely ...
I absolutly apreciated it, I will try to use it ...
Is seems a little bit complicated (for my level of course), I hope in the future somethings like this may be part of XPO...
Many Thanks ...
13 October, 2006
Geoff Davis
Sorry Oliver I just don't get it and I know it's my lack of knowledge based on this subject... Can you simply describe (I know, you thought you had already!) what Kenneth's problem was (I've read his post but it's still confusing to me, 'More datastore aware than DB aware', nope still don't get it) and what this does to overcome the problem in a more kinder-garden'ish style for me?

Nope, was going to have a guess but I'll think i'll leaver future questions to when I fully understand what this does... It's strange because I know i'm going to want to use, i'm in the process of writing a multi-user application using XPO over Remoting and I must get to grips with separating the XPO from the Remoting which i'm not doing!
13 October, 2006
Lu. Po.
About the xml example in "The RemotingTarget" chapter:
line 9 should be read: </xpopub:RemotingTarget> ?
line 10: maye should be above the actual line 9 ?
I'm reading the source ... and i'm learning somethings new about OOP and extensibility ... thank you ...
13 October, 2006
Alain
Hi Oliver.


Can you extend this to use in conjuction with RemObjects???

Regards, Alain
13 October, 2006
Reinhold
Hi Oliver,
I would like to second Alains request. Please extend to RO! :-)

Regards, Reinhold
14 October, 2006
Marco
Hi,

voting for RemObjects :)

thx, Marco
14 October, 2006
Oliver Sturm (DevExpress)
Lu. Po.: It looks complicated...
Sorry about that, it wasn’t supposed to. But it’s a pretty flexible system, so a description of just about everything it can do is bound to look complicated… So to give another example, kind of the simplest thing you can do, here’s another XML file:
... this is where the sample code was supposed to end up, but now it's not because CS doesn't let me put it here. See here for the sample.

Also, thanks for your hint about the mistake in the XML code, it's corrected now.

Geoff Davis: I have just posted a new article that should answer your questions. See here.

Several people: What about an extension for RemObjects?
We'll see... the good thing is, this should be possible externally - you could do it yourself (hint!). When I have the time, I might look into it for demonstrational purposes :-)
16 October, 2006
XPO
I was trying to stick this XML snippet in a comment on my article with the announcement of the XPO publication...
16 October, 2006
XPO

I was trying to stick this XML snippet in a comment on my article with the announcement of the XPO publication

28 March, 2008
XPO

There are no changes in this version -- surprising as it may be, nobody ever reported a single bug in

31 May, 2008
eXpress App Framework Team

This is post no. 10 in the mini series "10 exciting things to know about XAF". You can find

31 May, 2008

Please login or register to post comments.