XPO - A simple benchmark against EF 6 and EF Core

21 June 2018

See the second part here.


We recently published a BenchmarkDotNet-based benchmark on GitHub: https://github.com/DevExpress/XPO/tree/master/Benchmarks.

We used it to test the performance of the following Object-Relational Mapping (ORM) libraries for .NET Framework 4.6.1 and higher:

You can run the benchmark tests or review our results here. Needless to say, the lower the execution time the better. Example:

All benchmarks were executed using .NET 4.6.1, AnyCPU release builds (include warm-up), Windows 10 Enterprise x64, local Microsoft SQL Server 2016 Developer Edition, i7-7700 CPU @3.6GHz / 16GB RAM / SSD. Please note that DevExpress.Xpo and other referenced libraries will automatically be restored from Nuget. Edit the connection string in the App.config file, update the ORM library and target framework versions, if necessary. 

Your feedback is needed!

We kept the first version of this test as simple as possible.  Feel free to make data model and test case modifications to cover additional usage scenarios. For instance,  measure memory consumption, include scenarios with BLOBs, reference and collection properties, etc. We'd love to hear your feedback about this project. Drop us a line below, thanks.

See Also

XPO ORM Library – Available Free-of-Charge in v18.1!

Love XPO and want to help us promote it? Add the package through Nuget.org instead of DevExpress Nuget!

17 comment(s)

any comparison to nhibernate ?

22 June, 2018

Wow! Excellent work guys and thanks for making it free.

22 June, 2018
Dennis (DevExpress Support)


We do not plan to make comparison with NHibernate. You may find many comparisons on the Web. For instance, there is a good one at weblogs.asp.net/.../net-micro-orm-fetch-benchmark-results-and-the-fine-details:

"It’s quite obvious there are two frameworks which have a serious performance problem here: NHibernate and Entity Framework 6..."

@Scott: Thanks for your interest!

22 June, 2018
Juan Antonio Trujillo Montenegro

Why compare against EF Core 2.0 and not EF Core 2.1?

22 June, 2018
Dennis (DevExpress Support)

We did this comparison before the EF Core 2.1 announcement. Even though we do not expect significant result changes, we will consider updating our project. Thanks for your interest as well.

22 June, 2018
Martin Pelletier

You should promote XPO more. I know EF is more popular but should have given the option to use XPO instead EF with your scaffolding helpers.  

22 June, 2018
Paul Fuller

I use XPO already.  More interesting to me would be benchmarks of the alternative database targets for XPO such as SQL Server, PostgreSQL, SQLite, Firebird and good old MS-Access.  All should be run locally and with recommended settings.

The question is which of the target database engines gives the best performance across a range of activities and scenarios?  I've done a bit of testing and MS-Access wins by a factor of about 10x!

22 June, 2018

I'm a fan of EF6, I'd love to see the results after giving this to the EF6 team to ensure all the code used is the optimized approach. For example using Find instead of First for key based lookups. Using AsNoTracking, etc. I find EF to be very fast IF coded correctly per the task and the database is designed properly as well. I have never used XPO and don't intend to as I try to stick with main core technologies such as the widely adopted EF.

23 June, 2018
Stepan Lauryn

good news, good works.

24 June, 2018
Philippe Petrussa

Thanks for this comparison. Do you plan on publishing more real-world scenarios? Connections to Azure or WCS Services?

Thank you.

29 June, 2018
Dennis (DevExpress Support)

@Paul: We have not yet planned to compare performance of databases themselves (including SQL Azure). That is mainly because an ORM itself will not be tested much in this case. Also, comparing database performance often requires a much deeper database knowledge and often tuning to receive more fair results.

29 June, 2018
Dennis (DevExpress Support)

@Philippe: Would you please elaborate on your specific scenarios with Azure and WCF services as well as which ORM functionality you would like to test specifically and why? Feel free to attach your sample in the Support Center. We will be more than happy to review it and provide further comments.

29 June, 2018
Dennis (DevExpress Support)

@Neal: We appreciate your useful feedback. We'll call AsNoTracking() in a couple of tests where data is read and not modified further. It should slightly improve EF performance and we'll update our results page accordingly. It's also no question that EF is more widely adopted.

BTW, what is your experience with solving EF-related problems or questions that inevitably occur during development? Are StackOverFlow and other public community resources sufficient for this? I am asking this question because according to our surveys among XAF customers they call "DevExpress support" as the top factor when choosing XPO over EF.

29 June, 2018
Nikita Grigoryev

That performance comparison is not so evident. You should also compare ORM capabilities since they influence perfomance. EF6 is very "fat" ORM considering its capabilities so no wonder it's slow. It would be interesting to compare XPO and Linq2DB for example (which is considered to be THE fastest LINQ database access library).

3 July, 2018
Dennis (DevExpress Support)

@Nikita Grigoryev:

Like EF, XPO is not micro ORM. It is a powerful and mature ORM or persistence framework compared to EF by functionality. Learn more about main XPO features here (it even has the built-in security system actively used in XAF apps). Having said that, we think it is not appropriate to compare EF or XPO with LINQ to SQL, Dapper or any other lightweight libraries.

4 July, 2018
Philippe Petrussa

Thanks Dennis. As the title states, it is a simple benchmark and it only shows speed, not memory consumption. For Azure and WCF, I would like to see an example of an application consuming such services (e.g. a POS application accessing a central inventory system). Another element I think would be worth testing is Projections. In EF, if I'm not wrong, you can retrieve a subset of the fields of your entity that are updatable. In XPO, you cannot retrieve a projection as objects. This is an issue for me in a mail application I'm currently designing where I don't want to retrieve all the rows (including body, source and other fields with unlimited size). The mail app only displays the necessary fields for the list view and the preview window loads only the additional fields needed. Even with CollectionViewModel (EF ViewModel Wizard), the retrieve time is almost instantaneous and memory usage is low compared to XPO where it retrieves all the fields. I know I can use XPServerCollectionSource (XPO) but it basically retrieves fields I don't need for all the rows, just the one showing in the preview.

6 July, 2018
Dennis (DevExpress Support)

@Neal, Philippe: We have updated our test cases by the same link and blogged about the main changes at community.devexpress.com/.../xpo-a-simple-benchmark-against-ef-6-and-ef-core-update.aspx.

In particular, we have enabled the AsNoTracking option + added the new ProjectionLinq test case. As for updatable projections/XPServerCollectionSource, you can try to use the XPInstantFeedbackSource and XPView components instead (DataAccessMode = InstantFeedback or Dataview in XAF) + probably make some query tuning with the ExplicitLoading and Delayed attributes. Feel free to submit your test project to the Support Center if this does not help you achieve required level of performance.

17 July, 2018

Please login or register to post comments.