XPO - Interview with the Team

XPO Team Blog
30 August 2018

Many times, customers have asked us how we use XPO – or if we even do it at all. We decided to sit down with developers working on and with XPO and conduct an interview, with the goal of answering the most common questions. The result makes for an interesting read since it addresses many historical, strategical and technical aspects. Enjoy! 

When was the XPO Object/Relational Mapper released? Is it mature? 

XPO, the DevExpress ORM library for .NET, is a mature product that was first released in 2003. For around 15 years, XPO has been bundled with all our commercial product subscriptions. With the v18.1 release in May 2018, we made XPO available free of charge for everyone to use, as a strong alternative to Entity Framework Core. If you’re not familiar with XPO, you can learn more about its main feature set from the landing page. For a full list of XPO functionality, please visit our online documentation site, search our large support database or contact support. 

What platforms and technologies does XPO support?  

XPO helps access and manipulate data stored in-memory or within RDBMS database engines running on Windows, MacOS or Linux, in desktop, web and mobile projects powered by .NET. All .NET project types are supported by XPO, from the console to various ASP.NET flavors and UWP.  New .NET Standard-based technologies like Blazor will likely be supported in the future as well. We have also recently ported XPO from .NET to the VCL platform for Delphi and C++Builder apps - ExpressEntityMapping Framework

How actively does DevExpress use XPO internally? 

At DevExpress, we use XPO for nearly all our services: online documentation, the Support Center, our CRM, and the main website. You may not believe it, but even with the large number of version control options available to us (like GitHub, GitLab, and HG), XPO remains at the core of our internal version control system (DXVCS)! We still use our homegrown system because to date, no existing VCS has been able to handle the complexity of our product line/projects.  

In addition, XPO is at the core of several public DevExpress products. Our Business Application Framework XAF has used XPO for UI and database scaffolding since 2007. The XPO security engine is heavily used by the XAF security moduleDashboard and Reporting UI component libraries for various platforms use XPO APIs for database access (for instance,  in SqlDataSource and CachedReportSource). 

In short, we use XPO everywhere - from small to large mission critical apps. Oh, and our high-load systems work with databases ranging in size from 40 to 100 gigabytes.

Our Support Center:



Our internal version control system (DXVCS):
 

Are XPO team devs involved in internal product development? 

Yes, we are always involved – we want to see XPO succeed for our internal services and public DevExpress products. Real-life XPO based solutions help us continually improve the product, and to understand how it fits into a modern development landscape. For internal solutions the XPO team assists other teams to use the ORM correctly and efficiently. Company-wide we save resources with this unified approach, which means we don’t have to reinvent data access-related functionality whenever we do something new. 

How large is your internal app data model? Do you visualize it to better understand all the relations? 

Our largest internal app currently has 188 persistent classes. XPO supports a visual designer, but we don’t use it much. Our approach mainly a top-down (code first) strategy, which XPO was designed for traditionally. However, we work with many customers who use the visual designer extensively to visually maintain data models, and the XPO wizard to generate data models for existing databases.  

What about databases? Do you use SQL Server? 

We have used XPO with many SQL Server databases in production for 15 years - countless DevExpress customers rely on SQL Server. XPO also supports many other database systems. We don’t use them in production, but we know from our work with customers that Oracle, PostgreSQL, Firebird and MySql are the top database engines where SQL Server is not an option. Some customers successfully maintain XPO-based solutions that allow switching of the database server technology in deployment. 
Most XPO releases contain improvements to the level of support for one or more database systems. Check out our recent update as an example: it includes support for new driver versions and database specific features. 

Do you have hints for high-load applications with XPO? 

Since many of our own services handle quite high loads, we have had opportunity to test and evaluate different approaches in detail. Here are the three most helpful XPO features for such scenarios:  

  • The ThreadSafeDataLayer prevents data corruption to the metadata storage in XPDictionary when XPO is used in multi-threading scenarios. Code examples: one, two, three
  • The Connection Pool dynamically creates and destroys database connections depending on the current application load. Code examples: one, two, three
  • Object and Data Layer Caching enables caching on two different levels, returning valid object data and complete query results where possible, without executing queries on the server or even making a server roundtrip (SqlDependency is also supported). Code examples: one, two, three.

Each of these architectural components is highly configurable and can be fine-tuned to reflect your requirements. 

Do you have performance metrics for materialization, create, update, delete and other operations? 

Good performance is critical for our own services and we have always used performance tests with XPO and competing products to support our choice of XPO as a strategic platform. Recently we have published reproducible results of a comparison with EF6 and EF Core at https://github.com/DevExpress/XPO/tree/master/Benchmarks. These tests are based on the popular https://github.com/dotnet/BenchmarkDotNet benchmarking framework. 

Are there any internal projects that use other ORMs? 

Yes, there are several services that use Entity Framework 6, but these are small and deal with light loads only. Our internal team attempted to use Entity Framework for our Support Center as part of a large refactoring effort in 2014. However, they found several severe issues that ultimately prevented a move in that direction.  

A serious issue was the SQL generation mechanism used by Entity Framework. As an example, here is a  LINQ-query against the Northwind database. Its structure is typical for the Support Center, with a couple of joins and a subquery:

var query = from o in context.Orders 
        join od in context.Order_Details on o.OrderID equals od.OrderID 
        join c in context.Customers on o.CustomerID equals c.CustomerID into g 
        where g.Count() > 0 
        select new { o.OrderID, o.OrderDate, od.Quantity, od.UnitPrice, Count = g.Count() }; 


This is the SQL code XPO generates for the query. It is close to a SQL query that might be written manually by a developer. 

select N0."OrderID",N0."OrderDate",N1."Quantity",N1."UnitPrice", 
    (select count(*) as Res0 from "dbo"."Customers" N2  
where (N0."CustomerID" = N2."CustomerID"))  
from ("dbo"."Orders" N0 
    inner join "dbo"."Order Details" N1 on (N0."OrderID" = N1."OrderID")) 
where ((select count(*) as Res0 from "dbo"."Customers" N3  
where (N0."CustomerID" = N3."CustomerID")) > 0) and N0."OrderID" is not null 

On the other hand, Entity Framework generates this SQL:  

SELECT 
[Project1].[OrderID] AS [OrderID],  
[Project1].[OrderDate] AS [OrderDate],  
[Project1].[Quantity] AS [Quantity],  
[Project1].[UnitPrice] AS [UnitPrice],  
(SELECT  
    COUNT(1) AS [A1] 
    FROM [dbo].[Customers] AS [Extent4] 
    WHERE [Project1].[CustomerID] = [Extent4].[CustomerID]) AS [C1] 
FROM ( SELECT  
    [Extent1].[OrderID] AS [OrderID],  
    [Extent1].[CustomerID] AS [CustomerID],  
    [Extent1].[OrderDate] AS [OrderDate],  
    [Extent2].[UnitPrice] AS [UnitPrice],  
    [Extent2].[Quantity] AS [Quantity],  
    (SELECT  
        COUNT(1) AS [A1] 
        FROM [dbo].[Customers] AS [Extent3] 
        WHERE [Extent1].[CustomerID] = [Extent3].[CustomerID]) AS [C1] 
    FROM  [dbo].[Orders] AS [Extent1] 
    INNER JOIN [dbo].[Order Details] AS [Extent2] ON [Extent1].[OrderID] = [Extent2].[OrderID] 
)  AS [Project1] 
WHERE [Project1].[C1] > 0

In this case, the queries generated by XPO and EF executed in roughly the same time, even though their execution plans are a bit different. However, we also encountered EF generated queries that ran very slowly. Trying to optimize them by changing the LINQ query was almost impossible, since the generated SQL differs so much from the LINQ starting point. In many cases, our optimization efforts resulted in even more complex queries, and we gave up after a couple of weeks. 

A second major issue was that the Support Center uses SQL Server Full-Text Search technology to enable ticket search for our support engineers and DevExpress customers.  

We found only one method to integrate Full-Text Search with EF, by replacing some placeholders in the resulting SQL (https://stackoverflow.com/a/19644900/592926). That approach seemed questionable to us. 

In contrast, XPO provides a number of flexible extension points: 

  • Custom Criteria Language operators based on database-specific functions (example);
  • Custom functions and criteria for LINQ to XPO expressions (example); 
  • Custom connection providers for databases (example);
  • Decorator classes for various XPO layer interfaces: IObjectLayer, IDataLayer, IDataStore (example). 

We enabled  SQL Server Full-Text Search for XPO by means of a simple custom function, which can be used in any CriteriaOperator or XPQuery:

public class FTSContainsFunction : ICustomFunctionOperatorFormattable,   ICustomFunctionOperatorQueryable { 
    public object Evaluate(params object[] operands) { 
        return true; 
    } 
    string ICustomFunctionOperator.Name { 
        get { return "FTSContains"; } 
    } 
    public Type ResultType(params Type[] operands) { 
        return typeof(bool); 
    } 
    public string Format(Type providerType, params string[] operands) { 
        return String.Format("contains({0}, {1})", operands[0], operands[1]); 
    } 
    public System.Reflection.MethodInfo GetMethodInfo() { 
        return typeof(XpoExtensions).GetMethod("FTSContains"); 
    } 
}

XPO has a .NET Core version, too. Will you use this in new projects? 

We are very excited that XPO can run in so many different environments now, including Linux, iOS, Android, and UWP - .NET Core 2.0 and Xamarin/Mono make it possible (we even published a sample using Blazor and XPO). 

We tried to port XPO to .NET Standard 1.1, but there were too many restrictions at the time and we decided not to release it. In November 2017, we made a fully functional version of XPO available for .NET Standard 2.0. Since then, we added several features that are important for .NET Core projects, like Dependency Injection support for ASP.NET Core, .NET Core/.NET Standard project support in our ORM Data Model Wizard and compatibility with ASP.NET Core applications for the XPO Profiler.  See also our roadmap plans here.

Our internal team has already created several small services that use XPO for .NET Standard 2.0. Our experiences have been positive and we assume that more internal services will run on the platform in the near future. 

At this time,  XPO for .NET Core / .NET Standard 2.0 officially supports SQL Server, MySql, PostgreSQL, FireBird, SQLite and Oracle for .NET Core. The lack of support for other providers is not a limitation of XPO - only a function of support from RDBMS vendors. Once these vendors support .NET Standard 2.0, we will test XPO against these backend providers and add them to our supported list once our internal qualification requirements have been met. 

All in all, XPO for .NET Core is ready for production. It is 99% the same as XPO for the full .NET Framework (the 1% are some #ifdef lines), so you shouldn’t be worried about its quality or future.

What makes XPO unique? Which features do you value most for internal development? 

Our internal team supplied an interesting use case example of a very unique XPO feature: runtime data model generation without supporting design-time classes (example). The implementation is used in all our audit services, and it doesn’t use a single design-time persistent class. Metadata is generated at runtime based on a schema supplied by the main service. The dynamically generated auditable classes differ from the originals by including an extra modification time column as part of a compound key.

The ability to work with runtime persistent types is very popular among XPO and XAF customers. For instance,  the open-source eXpand Framework, a set of extensions for XAF, provides a visual constructor for UI and database schema: http://www.expandframework.com/#worldcreator  

The extensibility model offered by XPO is valued highly by our internal development teams. For example, we created an extension to insert a hint into a query (OPTION clause) to prevent SQL Server from using a bad execution plan in a specific case. 

It is easy to analyze the XPO query tree via a custom implementation of the IQueryCriteriaVisitor interface, because the code can work with CriteriaOperator instances in the SelectStatement object on the IDataStore level, instead of dealing with plain SQL.  

We overrode the ProcessSelectData method in a descendant of MSSqlConnectionProvider: 

protected override SelectStatementResult ProcessSelectData(SelectStatement selects) { 
    foreach(CriteriaOperator criteria in CollectCriteria(selects)) { 
        if(BadFunctionsFinder.Find(criteria).Count > 0) { 
            if(!string.IsNullOrEmpty(sqlOption)) { 
                postfixForRequest = string.Format("option ({0})", sqlOption); 
            } 
            break; 
        } 
    } 
    try { 
        return base.ProcessSelectData(selects); 
    } finally { 
        postfixForRequest = null; 
    } 
} 

For testing purposes, we rely very much on our InMemoryDataStore. This makes it possible to test the largest part of a system without using a real database and without functional restrictions.  EF Core now has a similar feature, but Entity Framework 6 does not support this approach out of the box. 

Finally, it is fantastic how easily the XPO data access layer can be changed: just modify the connection string. You can also switch from a local to a remote data store supplied by a service (WCF and others) by specifying a remote service URL instead of a database connection string (more details here).

How to get started with XPO? 

There are two ways to obtain XPO assemblies for your app: use either the Unified Installer or Nuget. Nuget is quick and easy but does not come with Visual Studio design time support. If you'd like to install XPO via the Unified Installer, simply download our trial installation and install our products in "trial mode." XPO will be automatically installed without trial restrictions. If you own a DevExpress paid subscription, you can use your existing DevExpress credentials when prompted by the Unified Installer. 

The free version of XPO does not include technical support. You can search our online documentationsupport database (with many Knowledge Base articles and user tickets) and numerous Code Example topics for free. Learning materials for beginners are located at DevExpress ORM Tool > Getting Started. You may want to watch Complete Tour of DevExpress ORM: eXpress Persistent Objects (XPO) - this YouTube video is old, but the same concepts apply today. 

There are also online demos (Console, Xamarin and ASP.NET Core) and offline demos that come with the unified installer. For instance, check these demo folders for different platforms: 

    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\CS\XpoTutorials\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\DevExpress.VideoRent.Win\CS\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\CS\ContactManagement\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\CS\GridMainDemo\ 
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WinForms\CS\SchedulerMainDemo\ 
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\ASP.NET\CS\ASPxGridViewDemos\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\ASP.NET\CS\Bootstrap.AspNetCore\ 
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\ASP.NET\CS\ASPxSchedulerDemos\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\WPF\DevExpress.VideoRent.Wpf\CS\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\eXpressApp Framework\XVideoRental\
    c:\Users\Public\Documents\DevExpress Demos 18.1\Components\eXpressApp Framework\MainDemo\

You can also get community help on StackOverFlow. If you want to receive technical support from DevExpress, you can purchase the XPO - ORM Library Subscription for $399.99. Full terms of use can be found here. For more information on this free version, check out this blog post.

Higher tier subscriptions like Universal, DXperience, WinForms, WPF, ASP.NET, Reporting, DevExtreme also include XPO tech support. Just like almost every business app needs database access APIs/ORM, it also needs good looking and reliable components for its UI. So, buying a DevExpress UI component subscription may be justified beyond just technical support for XPO.  

Thanks again for your business. We look forward to working with you now and into the future.

The DevExpress XPO Team
www.devexpress.com/support
XpoTeam@devexpress.com


Love XPO and want to help us promote it? Add the package through Nuget.org instead of DevExpress Nuget!

Free DevExpress Products - Get Your Copy Today

The following free DevExpress product offers remain available. Should you have any questions about the free offers below, please submit a ticket via the DevExpress Support Center at your convenience. We'll be happy to follow-up.
No Comments

Please login or register to post comments.