Oliver's Blog

November 2016 - Posts

  • DevExtreme Charts Data Handling Revisited with Open Source data-transformer

    In two blog posts, I recently mentioned data transformation requirements to enable binding of certain data structures to our DevExtreme Chart controls. To make these steps easier, I have now created an Open Source JavaScript library called data-transformer that provides utility functions for purposes similar to those outlined in my previous blog posts.

    The repository for data-transformer is here: https://github.com/oliversturm/data-transformer

    Detailed instructions for the use of the library can be found in the README file displayed on the repository front page. Feel free to use this in your own projects, and let me know if you find any problems!

    The first of the recent posts I'm referring to was Simulating per-series ChartJS data binding with a seriesTemplate. In the post, I was describing a nested 1:N master-detail data structure, and a helper function to flatten this to a simple list that can be bound to the chart. With the flattenOneToN helper from data-transformer, the same result can now be achieved like this:

      $("#chart").dxChart({ 
        dataSource: Array.from(dataTransformer.flattenOneToN(data)),
        commonSeriesSettings: {
        ...


    An updated version of the codepen from the original blog post is available to play with the code:



    The second of my recent posts was Binding an Array of Arrays with ChartJS, which described a data structure of nested arrays. Again, a more versatile function from data-transformer can be used to restructure this as a bindable format:

      $("#chart").dxChart({
        dataSource: Array.from(
          dataTransformer.iterableOfIterablesToObjects(data, ["days", "percent"])),
        rotated: true,
        ...

    The updated version of the related codepen is here:

  • Binding an Array of Arrays with ChartJS

    When I write applications these days, they usually use structured data (compare yesterday's post). This happens because it is very natural - my data is mostly retrieved from some kind of data storage, and whether that's a relational database that runs a query, or a document-based non-relational system (that's politically correct for NoSql), there is usually some structure involved. However, I have seen questions about using less formally structured data, namely arrays and arrays of arrays. The one advantage of this data structure that I can come up with is that it is very compact - certainly not easier to read for a human, but potentially faster to transfer if the data volume is large. Interestingly, some of our - ahem - competitors seem to focus on such data structures to the extent of excluding almost everything else. If you have existing chart setups that use array data, this post might also be interesting to you.


    As an example, here is an "array of arrays" data structure:

    const data = [[114, 8], [84, 17], [54, 31], [24, 17], [7, 2]];


    The problem you face when trying to bind this data to a ChartJS chart is that our binding system uses field names to determine which data elements are used for arguments and values (and potentially other series options). To bind the above data, you need to transform it into a shape that supplies field names. Fortunately this is not really hard to do, and I have written a flexible utility function for the purpose:

    function transformArrayOfArrays(source, ...names) {
      const defaultName = i => "field" + i;
      return source.map(a => {
         return a.reduce((r, v, i) => {
           r[i < names.length ? names[i] : defaultName(i)] = v;
           return r;
         }, {});
      });
    }

    The function accepts a source array as input, as well as - optionally - a list of field names. In the absence of field names, or if the nested arrays contain more values than there are specified names, the "defaultName" function is used to generate field names "field1", "field2" and so on. The function now generates its result by iterating over the top-level array elements. For each such element, the "reduce" function is used to create a new object and add the values from the nested array to this object, using the supplied or generated field names.

    Note: I had a question about the "new" ECMAScript 6 syntax I'm using in my sample code. I recommend you look into this if you write code in JavaScript! If you have trouble interpreting a piece of code I'm showing, I recommend checking out the CodePen samples I'm posting. If you follow the link to "edit on CodePen" and then click the "down-arrow" button in the upper right corner of the "JS (Babel)" panel, you have the option to see the compiled (or rather transpiled) JavaScript code. This can be very helpful when you get started with ES6.

    Using my sample data, I can now call the function like this:

    transformArrayOfArrays(data, "days", "percent")

    This sample call would transform my data to this structure:

    [{ days: 114, percent: 8 }, 
     { days: 84, percent: 17 },
     { days: 54, percent: 31 },
     { days: 24, percent: 17 }, 
     { days: 7, percent: 2 }]

    Of course this data can now be bound to a chart easily, using the same field names I supplied to the transformation function call:

      $("#chart").dxChart({
        dataSource: transformArrayOfArrays(data, "days", "percent"),
        series: [{
          argumentField: "days",
          valueField: "percent",
          type: "spline"
        }],
        .....
    


    I have created a sample that shows the use of the transformation helper in conjunction with a chart setup:


  • Simulating per-series ChartJS data binding with a seriesTemplate

    At this time, our DevExtreme Charts don't support per-series data binding. I believe this would be a useful feature to have because it simplifies certain solutions considerably, and I hear our team is considering adding the feature in the future. Meanwhile, I was researching approaches to simulate the effects of per-series binding, and here is a workaround you can apply if you need this right now.

    First, why does this requirement come up at all? I think the reason is mostly in the data structure you work with. When using data from structured storage, like a relational database system, your data would naturally be arranged along the lines of type separations in that storage.  As it happens, such normalized structures are also efficient for transfer, because they avoid redundancy. 

    Here is such a data structure:

    const data = [
        { place: "Los Angeles, CA, USA", values: [
            { month: "Jan", degreesC: 13.7 },
            { month: "Feb", degreesC: 14.2 },
            { month: "Mar", degreesC: 14.4 },
            { month: "Apr", degreesC: 15.6 },
            { month: "May", degreesC: 17.0 },
            { month: "Jun", degreesC: 18.7 },
            { month: "Jul", degreesC: 20.6 },
            { month: "Aug", degreesC: 21.3 },
            { month: "Sep", degreesC: 21.0 },
            { month: "Oct", degreesC: 19.3 },
            { month: "Nov", degreesC: 16.4 },
            { month: "Dec", degreesC: 13.8 }] },
        // .... more places and associated lists here
    ];


    With this data, you might want to create one chart series for each of the "places" on the top level, and then each series would be fed by the "values" array associated with the "place". If series-level binding were an option, you could write this:

    $("#chart").dxChart({ 
      series: data.map(p => ({  
        name: p.place,
        type: "line",
        data: p.values,
        argumentField: "month",
        valueField: "degreesC"
      }))
    });

    For clarity, please note that this does not work! Unfortunately, data cannot be bound to the series, it has to be bound to the chart on the top level. 

    In case you are unfamiliar with functional programming helpers like "map", you need to know that the helper "map" iterates over the list "data" and executes the function it is given for each element in that array. That function is shown here as a lambda expression, and it generates a series configuration object for each of the "place" elements in the data list, using the "place" name as the series name and assigning "place.values" to the hypothetical "data" property of the configuration object.

    Now that we've seen what the problem is, and how great the future could be if we had per-series binding, here's the workaround I promised. The basic idea is that we can use the "seriesTemplate" option of the chart configuration to make the chart generate series automatically on the basis of the data. This is a kind of meta-data-binding feature - instead of pre-creating series at design time and filling those with data at runtime, we set up a series template and use the chart-bound data to determine which series are created in the first place.


    Here is another example that does not quite work yet. It shows how the chart can be configured to accept the "seriesTemplate" option:

    $("#chart").dxChart({ 
      dataSource: data,
      commonSeriesSettings: {
        argumentField: "month",
        valueField: "degreesC",
        type: "line"
      },
      seriesTemplate: {
        nameField: "place"
      }
    });


    The "seriesTemplate" feature is clever: it watches the data field given in "nameField" and creates a new series automatically for each distinct value in that  field. The series is then configured by the parameters given in the "commonSeriesSettings" option, setting up the series field bindings correctly.


    The reason this setup doesn't work right now is that the chart expects all the fields mentioned in its configuration to be exist on the same "level" of the bound datasource. However, in the sample data only the "place" field exists on the top level, the other fields are nested in associated arrays. The remaining job is therefor to transform the data to a structure like this:

    [
        { place: "Los Angeles, CA, USA", month: "Jan", degreesC: 13.7 },
        ...
    ]


    To execute the transformation, I wrote this function:

    function transformData(data) {
      return data.map(p => 
        p.values.map(v => ({
          place: p.place, month: v.month, degreesC: v.degreesC
        }))).reduce((r, v) => r.concat(v));
    }


    The "map" helper is used again here, starting with the "data" list. For each element in that list, the lambda expression with the parameter "p" is executed ("p" is short for "place"). For each "place", another "map" is called on its "values" array, so eventually the inner lambda expression with the parameter "v" is executed for each of the value items in the nested arrays. This lambda expression returns a new object with the desired structure, combining the top-level and nested values. Finally, the "reduce" call is used to combine the individual arrays that have been generated per "place" into one - if you use a library like underscore or lodash, you can use a helper called "flatten" instead to achieve the same result.

    Now I only need to change the "dataSource" assignment of the chart to use this helper:

    dataSource: transformData(data),
    ...


    I have created a sample to demonstrate the approach described above:


  • Removing DevExtreme widgets from the DOM

    I was working on a few samples recently and stumbled upon this problem: when a widget created by our DevExtreme library was removed from the DOM, it wouldn't re-render itself on the same node later on.

    Initially I was using code like this to remove child nodes from the target node:

    while (node.lastChild) {
      node.removeChild(node.lastChild);
    }​

    This didn't work and after a bit of debugging, I tried to also reset node attributes that might have been added, in order to restore the target node to its original state:

    $.map(node.attributes, a => a.name).
      forEach(n => {
        if (n != "id") {
          node.removeAttribute(n)
        }
      });

    However, I was still unable to render a new widget in the same node later on. So the research continued, and with the help of our team I figured out that additional information is stored using the jQuery data() helper, and this also needs to be cleaned up properly. It is a good recommendation to call the _dispose() function on the widget to make sure any internal clean-up is performed at this point. Overall I ended up with this helper function to clean up the target node completely:

    function clearNode(node) {
      const chartData = $(node).data("dxChart");
      if (chartData) {
        chartData._dispose();
        $(node).removeData("dxChart");
      }
      const componentData = $(node).data("dxComponents");
      if (componentData) {
        $(node).removeData("dxComponents");
      }
      
      while (node.lastChild) {
        node.removeChild(node.lastChild);
      }
    
      $.map(node.attributes, a => a.name).
      forEach(n => {
        if (n != "id") {
          node.removeAttribute(n)
        }
      });
    }

    I have created a sample that shows the use of this code:

    Finally, note that the widgets always store their data in this way, regardless of the library you use to instantiate them. In other words, if you use our Knockout or Angular support, the data is stilled stored using the jQuery mechanism, and you would need similar steps if you were to manually remove a widget from a node.

LIVE CHAT

Chat is one of the many ways you can contact members of the DevExpress Team.
We are available Monday-Friday between 7:30am and 4:30pm Pacific Time.

If you need additional product information, write to us at info@devexpress.com or call us at +1 (818) 844-3383

FOLLOW US

DevExpress engineers feature-complete Presentation Controls, IDE Productivity Tools, Business Application Frameworks, and Reporting Systems for Visual Studio, along with high-performance HTML JS Mobile Frameworks for developers targeting iOS, Android and Windows Phone. Whether using WPF, ASP.NET, WinForms, HTML5 or Windows 10, DevExpress tools help you build and deliver your best in the shortest time possible.

Copyright © 1998-2017 Developer Express Inc.
All trademarks or registered trademarks are property of their respective owners