XPO - Interview with the Team

Many times, customers have asked us how we use XPO – or if we even do it at all. We decided to sit down with developers working on and with XPO and conduct an interview, with the goal of answering the most common questions. The result makes for an interesting read since it addresses many historical, strategical and technical aspects. Enjoy! 

When was the XPO Object/Relational Mapper released? Is it mature? 

XPO, the DevExpress ORM library for .NET, is a mature product that was first released in 2003. For around 15 years, XPO has been bundled with all our commercial product subscriptions. With the v18.1 release in May 2018, we made XPO available free of charge for everyone to use, as a strong alternative to Entity Framework Core. If you’re not familiar with XPO, you can learn more about its main feature set from the landing page. For a full list of XPO functionality, please visit our online documentation site, search our large support database or contact support. 

What platforms and technologies does XPO support?  

XPO helps access and manipulate data stored in-memory or within RDBMS database engines running on Windows, MacOS or Linux, in desktop, web and mobile projects powered by .NET. All .NET project types are supported by XPO, from the console to various ASP.NET flavors and UWP.  New .NET Standard-based technologies like Blazor will likely be supported in the future as well. We have also recently ported XPO from .NET to the VCL platform for Delphi and C++Builder apps - ExpressEntityMapping Framework

How actively does DevExpress use XPO internally? 

At DevExpress, we use XPO for nearly all our services: online documentation, the Support Center, our CRM, and the main website. You may not believe it, but even with the large number of version control options available to us (like GitHub, GitLab, and HG), XPO remains at the core of our internal version control system (DXVCS)! We still use our homegrown system because to date, no existing VCS has been able to handle the complexity of our product line/projects.  

In addition, XPO is at the core of several public DevExpress products. Our Business Application Framework XAF has used XPO for UI and database scaffolding since 2007. The XPO security engine is heavily used by the XAF security moduleDashboard and Reporting UI component libraries for various platforms use XPO APIs for database access (for instance,  in SqlDataSource and CachedReportSource). 

In short, we use XPO everywhere - from small to large mission critical apps. Oh, and our high-load systems work with databases ranging in size from 40 to 100 gigabytes.

Our Support Center:



Our internal version control system (DXVCS):
 

Are XPO team devs involved in internal product development? 

Yes, we are always involved – we want to see XPO succeed for our internal services and public DevExpress products. Real-life XPO based solutions help us continually improve the product, and to understand how it fits into a modern development landscape. For internal solutions the XPO team assists other teams to use the ORM correctly and efficiently. Company-wide we save resources with this unified approach, which means we don’t have to reinvent data access-related functionality whenever we do something new. 

How large is your internal app data model? Do you visualize it to better understand all the relations? 

Our largest internal app currently has 188 persistent classes. XPO supports a visual designer, but we don’t use it much. Our approach mainly a top-down (code first) strategy, which XPO was designed for traditionally. However, we work with many customers who use the visual designer extensively to visually maintain data models, and the XPO wizard to generate data models for existing databases.  

What about databases? Do you use SQL Server? 

We have used XPO with many SQL Server databases in production for 15 years - countless DevExpress customers rely on SQL Server. XPO also supports many other database systems. We don’t use them in production, but we know from our work with customers that Oracle, PostgreSQL, Firebird and MySql are the top database engines where SQL Server is not an option. Some customers successfully maintain XPO-based solutions that allow switching of the database server technology in deployment. 
Most XPO releases contain improvements to the level of support for one or more database systems. Check out our recent update as an example: it includes support for new driver versions and database specific features. 

Do you have hints for high-load applications with XPO? 

Since many of our own services handle quite high loads, we have had opportunity to test and evaluate different approaches in detail. Here are the three most helpful XPO features for such scenarios:  

  • The ThreadSafeDataLayer prevents data corruption to the metadata storage in XPDictionary when XPO is used in multi-threading scenarios. Code examples: one, two, three
  • The Connection Pool dynamically creates and destroys database connections depending on the current application load. Code examples: one, two, three
  • Object and Data Layer Caching enables caching on two different levels, returning valid object data and complete query results where possible, without executing queries on the server or even making a server roundtrip (SqlDependency is also supported). Code examples: one, two, three.

Each of these architectural components is highly configurable and can be fine-tuned to reflect your requirements. 

Do you have performance metrics for materialization, create, update, delete and other operations? 

Good performance is critical for our own services and we have always used performance tests with XPO and competing products to support our choice of XPO as a strategic platform. Recently we have published reproducible results of a comparison with EF6 and EF Core at https://github.com/DevExpress/XpoNetCoreDemos/tree/master/ORMBenchmark. These tests are based on the popular https://github.com/dotnet/BenchmarkDotNet benchmarking framework. 

Are there any internal projects that use other ORMs? 

Yes, there are several services that use Entity Framework 6, but these are small and deal with light loads only. Our internal team attempted to use Entity Framework for our Support Center as part of a large refactoring effort in 2014. However, they found several severe issues that ultimately prevented a move in that direction.  

A serious issue was the SQL generation mechanism used by Entity Framework. As an example, here is a  LINQ-query against the Northwind database. Its structure is typical for the Support Center, with a couple of joins and a subquery:

var query = from o in context.Orders 
        join od in context.Order_Details on o.OrderID equals od.OrderID 
        join c in context.Customers on o.CustomerID equals c.CustomerID into g 
        where g.Count() > 0 
        select new { o.OrderID, o.OrderDate, od.Quantity, od.UnitPrice, Count = g.Count() }; 


This is the SQL code XPO generates for the query. It is close to a SQL query that might be written manually by a developer. 

select N0."OrderID",N0."OrderDate",N1."Quantity",N1."UnitPrice", 
    (select count(*) as Res0 from "dbo"."Customers" N2  
where (N0."CustomerID" = N2."CustomerID"))  
from ("dbo"."Orders" N0 
    inner join "dbo"."Order Details" N1 on (N0."OrderID" = N1."OrderID")) 
where ((select count(*) as Res0 from "dbo"."Customers" N3  
where (N0."CustomerID" = N3."CustomerID")) > 0) and N0."OrderID" is not null 

On the other hand, Entity Framework generates this SQL:  

SELECT 
[Project1].[OrderID] AS [OrderID],  
[Project1].[OrderDate] AS [OrderDate],  
[Project1].[Quantity] AS [Quantity],  
[Project1].[UnitPrice] AS [UnitPrice],  
(SELECT  
    COUNT(1) AS [A1] 
    FROM [dbo].[Customers] AS [Extent4] 
    WHERE [Project1].[CustomerID] = [Extent4].[CustomerID]) AS [C1] 
FROM ( SELECT  
    [Extent1].[OrderID] AS [OrderID],  
    [Extent1].[CustomerID] AS [CustomerID],  
    [Extent1].[OrderDate] AS [OrderDate],  
    [Extent2].[UnitPrice] AS [UnitPrice],  
    [Extent2].[Quantity] AS [Quantity],  
    (SELECT  
        COUNT(1) AS [A1] 
        FROM [dbo].[Customers] AS [Extent3] 
        WHERE [Extent1].[CustomerID] = [Extent3].[CustomerID]) AS [C1] 
    FROM  [dbo].[Orders] AS [Extent1] 
    INNER JOIN [dbo].[Order Details] AS [Extent2] ON [Extent1].[OrderID] = [Extent2].[OrderID] 
)  AS [Project1] 
WHERE [Project1].[C1] > 0

In this case, the queries generated by XPO and EF executed in roughly the same time, even though their execution plans are a bit different. However, we also encountered EF generated queries that ran very slowly. Trying to optimize them by changing the LINQ query was almost impossible, since the generated SQL differs so much from the LINQ starting point. In many cases, our optimization efforts resulted in even more complex queries, and we gave up after a couple of weeks. 

A second major issue was that the Support Center uses SQL Server Full-Text Search technology to enable ticket search for our support engineers and DevExpress customers.  

We found only one method to integrate Full-Text Search with EF, by replacing some placeholders in the resulting SQL (https://stackoverflow.com/a/19644900/592926). That approach seemed questionable to us. 

In contrast, XPO provides a number of flexible extension points: 

  • Custom Criteria Language operators based on database-specific functions (example);
  • Custom functions and criteria for LINQ to XPO expressions (example); 
  • Custom connection providers for databases (example);
  • Decorator classes for various XPO layer interfaces: IObjectLayer, IDataLayer, IDataStore (example). 

We enabled  SQL Server Full-Text Search for XPO by means of a simple custom function, which can be used in any CriteriaOperator or XPQuery:

public class FTSContainsFunction : ICustomFunctionOperatorFormattable,   ICustomFunctionOperatorQueryable { 
    public object Evaluate(params object[] operands) { 
        return true; 
    } 
    string ICustomFunctionOperator.Name { 
        get { return "FTSContains"; } 
    } 
    public Type ResultType(params Type[] operands) { 
        return typeof(bool); 
    } 
    public string Format(Type providerType, params string[] operands) { 
        return String.Format("contains({0}, {1})", operands[0], operands[1]); 
    } 
    public System.Reflection.MethodInfo GetMethodInfo() { 
        return typeof(XpoExtensions).GetMethod("FTSContains"); 
    } 
}

XPO has a .NET Core version, too. Will you use this in new projects? 

We are very excited that XPO can run in so many different environments now, including Linux, iOS, Android, and UWP - .NET Core 2.0 and Xamarin/Mono make it possible (we even published a sample using Blazor and XPO). 

We tried to port XPO to .NET Standard 1.1, but there were too many restrictions at the time and we decided not to release it. In November 2017, we made a fully functional version of XPO available for .NET Standard 2.0. Since then, we added several features that are important for .NET Core projects, like Dependency Injection support for ASP.NET Core, .NET Core/.NET Standard project support in our ORM Data Model Wizard and compatibility with ASP.NET Core applications for the XPO Profiler.  See also our roadmap plans here.

Our internal team has already created several small services that use XPO for .NET Standard 2.0. Our experiences have been positive and we assume that more internal services will run on the platform in the near future. 

At this time,  XPO for .NET Core / .NET Standard 2.0 officially supports SQL Server, MySql, PostgreSQL, FireBird, SQLite and Oracle for .NET Core. The lack of support for other providers is not a limitation of XPO - only a function of support from RDBMS vendors. Once these vendors support .NET Standard 2.0, we will test XPO against these backend providers and add them to our supported list once our internal qualification requirements have been met. 

All in all, XPO for .NET Core is ready for production. It is 99% the same as XPO for the full .NET Framework (the 1% are some #ifdef lines), so you shouldn’t be worried about its quality or future.

What makes XPO unique? Which features do you value most for internal development? 

Our internal team supplied an interesting use case example of a very unique XPO feature: runtime data model generation without supporting design-time classes (example). The implementation is used in all our audit services, and it doesn’t use a single design-time persistent class. Metadata is generated at runtime based on a schema supplied by the main service. The dynamically generated auditable classes differ from the originals by including an extra modification time column as part of a compound key.

The ability to work with runtime persistent types is very popular among XPO and XAF customers. For instance,  the open-source eXpand Framework, a set of extensions for XAF, provides a visual constructor for UI and database schema: http://www.expandframework.com/#worldcreator  

The extensibility model offered by XPO is valued highly by our internal development teams. For example, we created an extension to insert a hint into a query (OPTION clause) to prevent SQL Server from using a bad execution plan in a specific case. 

It is easy to analyze the XPO query tree via a custom implementation of the IQueryCriteriaVisitor interface, because the code can work with CriteriaOperator instances in the SelectStatement object on the IDataStore level, instead of dealing with plain SQL.  

We overrode the ProcessSelectData method in a descendant of MSSqlConnectionProvider: 

protected override SelectStatementResult ProcessSelectData(SelectStatement selects) { 
    foreach(CriteriaOperator criteria in CollectCriteria(selects)) { 
        if(BadFunctionsFinder.Find(criteria).Count > 0) { 
            if(!string.IsNullOrEmpty(sqlOption)) { 
                postfixForRequest = string.Format("option ({0})", sqlOption); 
            } 
            break; 
        } 
    } 
    try { 
        return base.ProcessSelectData(selects); 
    } finally { 
        postfixForRequest = null; 
    } 
} 

For testing purposes, we rely very much on our InMemoryDataStore. This makes it possible to test the largest part of a system without using a real database and without functional restrictions.  EF Core now has a similar feature, but Entity Framework 6 does not support this approach out of the box. 

Finally, it is fantastic how easily the XPO data access layer can be changed: just modify the connection string. You can also switch from a local to a remote data store supplied by a service (WCF and others) by specifying a remote service URL instead of a database connection string (more details here).

How to get started with XPO? 

There are two ways to obtain XPO assemblies for your app: use either the Unified Installer or Nuget. Nuget is quick and easy but does not come with Visual Studio design time support. If you'd like to install XPO via the Unified Installer, simply download our trial installation and install our products in "trial mode." XPO will be automatically installed without trial restrictions. If you own a DevExpress paid subscription, you can use your existing DevExpress credentials when prompted by the Unified Installer. 

The free version of XPO does not include technical support. You can search our online documentationsupport database (with many Knowledge Base articles and user tickets) and numerous Code Example topics for free. Learning materials for beginners are located at DevExpress ORM Tool > Getting Started. You may want to watch Complete Tour of DevExpress ORM: eXpress Persistent Objects (XPO) - this YouTube video is old, but the same concepts apply today. 

There are also online demos at https://github.com/DevExpress/XpoNetCoreDemos (Console, Xamarin and ASP.NET Core) and offline demos that come with the unified installer. For instance, check these demo folders for different platforms: 

    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\CS\XpoTutorials\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\DevExpress.VideoRent.Win\CS\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\CS\ContactManagement\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\CS\GridMainDemo\ 
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\CS\SchedulerMainDemo\ 
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\ASP.NET\CS\ASPxGridViewDemos\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\ASP.NET\CS\Bootstrap.AspNetCore\ 
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\ASP.NET\CS\ASPxSchedulerDemos\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WPF\DevExpress.VideoRent.Wpf\CS\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\eXpressApp Framework\XVideoRental\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\eXpressApp Framework\MainDemo\

You can also get community help on StackOverFlow. If you want to receive technical support from DevExpress, you can purchase the XPO - ORM Library Subscription for $399.99. Full terms of use can be found here. For more information on this free version, check out this blog post.

Higher tier subscriptions like Universal, DXperience, WinForms, WPF, ASP.NET, Reporting, DevExtreme also include XPO tech support. Just like almost every business app needs database access APIs/ORM, it also needs good looking and reliable components for its UI. So, buying a DevExpress UI component subscription may be justified beyond just technical support for XPO.  

Thanks again for your business. We look forward to working with you now and into the future.

The DevExpress XPO Team
www.devexpress.com/support
XpoTeam@devexpress.com


Love XPO and want to help us promote it? Add the package through Nuget.org instead of DevExpress Nuget!

21 comment(s)
cedrus bank

It's almost 2 AM here and I'm enjoying reading every bit of this interview.

I speak positively and enthusiastically about DevExpress products and during any discussion, I talk first about XPO as for me it is the best DevExpress product. I just can't live without it any more .

So thank you XPO team for the product and your huge efforts.

Regards,

Bilal Harb

30 August, 2018
renejdm

Thank you for this blog. I will take another look at using XPO.

30 August, 2018
Abdu Bukres 3

Instead of mentioning your internal apps which none of us has access to, I would have preferred if you have mentioned complete apps on Github which use XPO.

30 August, 2018
Bassam Abdelaal

Before, I tried LINQ to SQL , EF , Telerik Data Access , then XPO , I can say I found XPO far superior than all, specially EF , XPO syntax is much easier , great features , work very well with DX controls , performance is faster , really faster in our developed ERP in many cases , we use it in traditional .net and with XAF , that is (XAF) by much far , the best platform I used in programming for 20 years, I call it the developer’s dream !

So thank you DevExpress for those two amazing products , XPO + XAF , keep it up

31 August, 2018
Dennis (DevExpress Support)

@Bilal: Thank you for your warm words!

@renejdm: Thank you for your interest in our ORM library!

@Abdu Bukres: Me too! Unfortunately, not many internal business apps or public commercial products are available on GitHub regardless of ORM libraries they use. Would you please clarify what you would like to see or check in these complete apps? I believe we will be able to figure out an alternative way to address your problem or requirement. Thanks in advance!

31 August, 2018
Brendon Muck [DevExpress MVP]

@Abdu: The WinForms Video Rental Demo application that is installed with the WinForms subscription is built with XPO. You can find the C# & VB.NET source here:

C:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\DevExpress.VideoRent.Win

31 August, 2018
Dennis (DevExpress Support)

@Brendon: Thank you. Yes, this is a good reference application. There are more links to demos in the end of my article, if this is what Abdu is looking for.

31 August, 2018
Neven

I have been using XPO for 10 years now in all of my projects. You did a great job. One issue remains ;), please provide an example application - Outlook inspired app, or make a new one - of XPO in MVVM framework.

31 August, 2018
Junior Thurler

Hi Dennis! How many people work at XPO team? Is there any plans to put the project open source?

I believe that DevExpress should develop more training courses about your products, XPO and XAF are two cases that you should publish some courses at PluralSight or another similar site.

In Pluralsight for example, we have just one course about CodeRush and seven training about Telerik controls!

Best regards,

31 August, 2018
Dennis (DevExpress Support)

@Neven: Thank you for your warm words and suggestions!

@Junior: We have 2 developers, 3 support developers, 2 tech writers. I've attached a photo of our team in the end for your reference.

I don't expect that XPO will ever be made open source nor do we expect to publish its source code free-of-charge. Things can change and we'll certainly take everyone's feedback into consideration.

More training courses is definitely something I would also love to have for XAF and XPO too.

31 August, 2018
Gosha

In general DX products are great. Tried couple major competitor products.... not even near with support and quality...

Great job guys!

31 August, 2018
Dmitri Peredera

In general, XPO is great.

But some kind of migrations like in EF Core would be nice as XPO isn't very good at managing db scheme.

Because sometimes one look at the db and see the table, but there is no code associated with it any-more and deleting tables manually isn't that easy.

Basically, in order to live long and happy with XPO we save in comments:

- store all table names ever used

- store all columns in table and types ever used.

Because if you try to update 2-3 y.o. app and use the same name debugging the problem will be "fun".

1 September, 2018
Sigurd Decroos _

I use XPO everywhere, it is by far the best and in most cases the fastest too.

But... Add support for spatial datatypes!

2 September, 2018
B Jansen

On your blog you mentioned XPO for Delphi multiple times. Coming from Delphi I'm curious about this. Is there some documentation available? Will it be open sourced just like the .net version?

3 September, 2018
José Enrique

I hope this kind of post could help to make more popular XPO to a wide audience, especially since it is free and the best ORM!

By the way, take a look at the last questions format. I think you should seriously consider using markdown and integrate it into DevExpress rich editors component to avoid that kind of problem. It is a trend for some reason.

4 September, 2018
Dennis (DevExpress Support)

@All: Thank you for the faith you put in DevExpress and XPO in particular.

@Dmitri: We would also love to have this feature. Not promising anything, but we hope to reconsider its large cost vs value for the next year.

@Sigurd: We have discussed this at

www.devexpress.com/.../regarding-devexpress-xpo-geo-spatial-types.

@B Jansen: Sure, you can download the documentation by one of these links:

go.devexpress.com/Documentation_VCL_CHM_VCLSubscription.aspx

go.devexpress.com/Documentation_VCL_PDF_VCLSubscription.aspx (look for the ExpressEntityMappingFramework file in there). And a small correction - XPO is NOT open source, just free to use.

@Jose Enrique: We hope so too. BTW, do you mean blog comments, Support Center tickets or where do you need the markdown?

4 September, 2018
Clinton Dale

Totally dependent on the diagram tool and its generation of classes.  I'm saddened that you don't use the GUI internally, since that may mean less work is done on it.

15 September, 2018
Uriah (DevExpress Support)

@Clinton: We measure the component/feature importance based on its popularity among our customers. Therefore, the fact that our developers prefer to manually declare classes does not affect the ORM Data Model Wizard/Designer development. We plan to continue developing it, because we see that our customers need this component.

See also our latest blog post where we desribed new XPO features: https://community.devexpress.com/blogs/xpo/archive/2018/09/14/xpo-what-you-can-expect-in-mid-november-v18-2.aspx

17 September, 2018
developer_paul

Its great to see XPO being worked on and improved again. As a long time customer and having used XPO for many years I think it is a fantastic ORM. Our application <a href="https://www.stockportfolioorganizer.com">https://www.stockportfolioorganizer.com</a> was built entirely using DevExpress components and XPO with well over 100 persistent classes.  Many different ORM's such as EF and NHibernate were evaluated before settling on XPO. The problem with other frameworks is that they work fine for simple stuff but it's only when the complexity increases that you start to see major problems with them.

17 September, 2018
developer_paul

Its great to see XPO being worked on and improved again. As a long time customer and having used XPO for many years I think it is a fantastic ORM. Our application www.stockportfolioorganizer.com was built entirely using DevExpress components and XPO with well over 100 persistent classes.  Many different ORM's such as EF and NHibernate were evaluated before settling on XPO. The problem with other frameworks is that they work fine for simple stuff but it's only when the complexity increases that you start to see major problems with them.

17 September, 2018
akram abdelaziz

Great tools and makes life easier and many scenarios supported :)

2 October, 2018

Please login or register to post comments.