Work with built-in modules

During the time of designing the data handling process and handler creation, it may be required for the user to connect and use support libraries. Besides the main built-in libraries, the modules are added to the architectural data processing layer, representing by themselves the additional libraries that can be useful by the data processing algorithm creation to optimize it. Each of these modules exhibits a rich and convenient enough set of features, which facilitates the development process.

The built-in libraries must be specially requested. The function of importing the built-in modules is created in order to get rid of a necessity to access redundant resources in the process of launching and executing an algorithm set in the handler. Those resources that were invoked, but which are not used, can slow down the process of handling, which is better to try to avoid.

The invocation of any built-in module is implemented in the following way. Initially, it is necessary to create a new variable of the  const datatype. Following that, the function to the request should be assigned to the variable, in which the name of the required module or a pointer to it is indicated:

const m = require('name_of_module');

In the source code development process for a handler, the next modules can be added:

The examples of loading and using the built-in modules are fed in the repository of GitHub.

Math.js

Math.js – is the extended library that is publicly available and used for JavaScript and Node.js. It offers a wide range of built-in functions and mathematical constants, allowing to produce a huge complex of various mathematical operations on large numerical data, complex numbers, fractions, matrices, etc. The library opportunities are quite extensive that allow facilitating the mathematical computations during the algorithm execution.

In the data handling process, it is often required to calculate some new value the data by which were not received directly from the device. Therefore, the application of this built-in module in some cases can simplify the handling process.

In order to load this built-in module, it should be called globally or locally in the function:

const math = require('mathjs');

Next, access to the library is implemented via the  math variable.

After the module invocation, all functions built into the current library can be used.

For example, a derivative of some value should be computed by the data processing. Then a new handler should be created. It needs to:

  • load the math.js library ;
  • compute the derivative ;
  • assign it to the variable of the let, data type, the value of which can be changed ;
  • substitute the input variable value in the resulting expression ;
  • convert the acquired object data format to a string for output and obtaining the eventual result.
// calculates a derivative

const math = require('mathjs');

function process(a) {
  let derivative = math.derivative('a^3 + 4*a^2 + 7', 'a');     //3*a^2 + 8*a
  derivative = derivative.eval({a});
  const da = derivative.toString();
  return { da };
}

Buffer

This module represents the  Buffer, class, which is a part of API Node.js, that allows you to work and interact with flows of different binary data. This class works with objects of the TypedArray, data type, which are visually similar to arrays of the binary data but are applied precisely by working with buffers. The Buffer module may be useful by data processing precisely because of working with flows. Indeed, when the question is the data flow, it is understood that sending and movement of data have occurred. For instance, it is often necessary to transfer data for processing, make them available for reading. In those cases, as a rule, queues of data packets requested for the processing appear. This queue is buffered.

The buffer represents a certain part of the random access memory in which the data packets are accumulated. They are lined up in accordance with the time of sending a request and sent one after another to the output for processing. Node.js technologies are designed to create and send a request. However, they cannot control the data flows. Buffers are already used for this purpose.

By working with the Internet of Things technologies, it may be necessary to process data by using the special protocol. For example, the data can be sent from the device by the protocol as a sequence of hexadecimal numbers. They require further processing, in particular, the parsing. The data may accumulate and wait for processing. For building and managing the queue, the handler may be used inside which the work with the buffer is realized.

It starts by invoking the built-in Buffer module:

const { Buffer } = require('buffer');

The buffer is created inside the function. By its creation, the buffer content, data encoding, the buffer dimension (number of bytes) can be indicated in the input data. In the considering example, by two buffers creation, the input data, i.e., the values of certain model parameters, are initially written to them. Next, their dimensions are summed up. After that, the buffers are combined into one. To correctly display the buffer contents, it must be decoded.

// creates two buffers and combines them

const { Buffer }  = require('buffer');

function process(a, b) {
  const buf_a = Buffer.from(a);
  const buf_b = Buffer.from(b);
  const totalLength = buf_a.length + buf_b.length;
  const buf_sum = Buffer.concat([buf_a, buf_b], totalLength).toString();
  return { buf_sum };
}

History

The built-in history module allows you to manage the history and collect the last data packets. It returns an array of objects containing only data sent earlier, not including the current packet. The quantity of packets, which can be learned from history, is equal to 50 by default. However, if necessary, the user is able to specify the needed number of packets by himself. The history module can be useful by data analysis, by comparing the current data with the previous, for specific calculations based on previous values of the parameters, etc.

Initially, the module should be invoked. It is worth considering that the call of the built-in reject, history, and split modules is implemented through the @ric/handler/ pointer. For example,

const history = require('@ric/handler/history');

Following that, the data packets are loaded from history. To unload several data packets, in the history.get() method are specified one or several variables that correspond to a specific input or output parameters of the function, and the number of uploaded packets, which is the same for all specified parameters. The history.last() method allows uploading only the last data packet stored in the history. At the same time, any of the specified functions may return an empty array of objects if the history was not written yet for the pointed parameter at the current moment.

The input data for the function given in the example of using the history.get() method is the a variable. A certain parameter of the object model corresponds to it. In the specified example, the arithmetic mean of ten a variable values is calculated. Therefore, it may be considered that the numeric argument value is assigned to it. Accordingly, nine last packets are uploaded from the history for that parameter. Next, the arithmetic mean of the numeric argument values is calculated based on the current one and the previous nine values.

// calculates the average value

const history = require('@ric/handler/history');

function process(a) {
  const packets = history.get(['a'], 10);
  let avg = a;
  for (let i = 0; i < packets.length; i++) {
      avg += packets[i];
  }
  avg /= (packets.length + 1)
  return { avg }
}

The speed value is calculated in the example of using the history.last() method, the scalar value of which is computed as the displacement difference divided by time. In this case, the time variable is the default parameter, and its value is fixed. The displacement difference is calculated as the difference between the current and the previous value. There checks for the presence of the field in the object. If it is and the object is not empty, then the value stored in the last packets of the current parameter in the history is assigned to the prevS variable. If the object is empty and the history was not saved yet, then this variable value will remain equal to zero.

// calculates a speed

const history = require('@ric/handler/history');

function process(t = 5, s) {
  let prevS = 0;  
  const last = history.last(['s']);
  if ('s' in last) prevS = +last.s;
  const v = (s - prevS) / t;
  return { v };
}

The history module allows uploading an array of objects. Accordingly, it should be remembered that access to every loaded value is implemented as to the array element. At the same time, the features of work with the objects should be noted.

Reject

The reject module is built in the handler in such case if such data packet is found in the queue of processed packets, handling of which cannot be executed. In this case, the packet is discarded from the queue, and its processing will not be produced. The processing algorithm will be applied to the next packet as soon as it is received. The processing operation cancel is executed based on the condition specified in the source code. This module is widely used enough. It can be applied for filtration realization, actual data acquisition, old data discard, etc.

For instance, the filtration is realized in the following way. Sometimes, incorrect data may be received from sensors. For example, the distance sensor may send a negative value of the measured distance. At the same time, the distance passed by the object should be applied for further handling. In this case, negative values will skew the results of calculations, and the data processing will be implemented incorrectly. Accordingly, the negative values of the passed distance must be discarded by applying the reject() function. The non-negative current values are written in an array that can be transferred to further processing.

// makes validation of data and reject if they does not meet the specified condition

const reject = require('@ric/handler/reject');

function process(d) {
  const distance = [];
  if (d < 0) return reject();
  distance.push(d);
  return { distance };
}

Split

In some cases, it may be necessary to transmit obtained data in the form of several separate packets through different channels by data handling. In order to do this, the split module may be used. This allows breaking the output data into several packets. For example, as a result of executing the algorithm prescribed in the handler, the output should be data array. If applying the split() function to this array, then several data packets will be formed, the quantity of which will be equal a length of an obtained array, at the same time, a separate element of the array will be saved in each packet.

For instance, the value of an effective temperature should be computed for twenty last data packets. The effective temperature is calculated by the formula:

ЭТ = t - 0,4(t - 10)(1 - f/100),

where ЭТ - effective temperature;   
t - air temperature;  
f - air humidity.

It is necessary to compute the value of the effective temperature for the twenty last packets. As a result, an array should be formed, which must be split into separate data packets on the function output in order to use every element of the array separately. In this case, the array realization and filling algorithm is prescribed in the function. It is worth noting that data uploaded from history can have various data types. Therefore, for numerical calculations, the data type should be explicitly specified for every element of uploaded arrays. As a result, the function returns split() in the end which means, in this case, the transmission of the twenty packets on the output.

// splits the data

const history = require('@ric/handler/history');
const split = require('@ric/handler/split');

function process(temp, humid) {
  const temp_last = history.get(['temp'], 20);
  const humid_last = history.get(['humid'], temp_last.length);
  const eff_temp = [];
  for (let i = 0; i < temp_last.length; i++)
    eff_temp.push(+temp_last[i] - 0.4*(+temp_last[i] - 10)*(1 - +humid_last[i]/100));
  return split(eff_temp);
}

We use cookies to improve your experience. Let’s accept it and move on.