Lab 12 - Thingspeak platform

From ThingSpeak/MathWorks documentation:

Set-up

Create a Thingspeak account.
Note: Free accounts are limited to 4 channels and the maximum rate of sending data is at 15 second intervals

Part 1 - Sending data to Channels

In ThingSpeak, data is organized into Channels. A channel contains several fields. Fields hold the values (numeric or non-numeric). For example, you may have something like:

• Let's first set up a new channel. Follow the official tutorial.
  • Set field 1 to have name "Magnetic sensor"

Data can be written & read to/from a channel via HTTP or MQTT. The HTTP supports several ways of sending data.

The most basic way of sending data to Thingspeak:

• Try out sending numeric values to field 1 by making the request from your browser (or use Postman, cURL, etc if you prefer).

After Sending data, you should notice the values appear in the Channels Private/Public view Thingspeak visualization:

Using the ESP32 in-built Hall Sensor

Let's write some Arduino code to read values from the hall effect sensor that is built in to ESP32. This sensor measures magnitude of magnetic field.

• To get a feel for how the sensor works, it's recommended to first try out the pre-existing Arduino IDE example: File -> Examples -> ESP32 -> HallSensor
  • If you don't have magnets at hand (e.g. a fridge magnet), you can for example use the speaker of your smartphone as a magnet. However, since such a small speaker is a rather weak magnet and the Hall Sensor can be quite noisy, you should try taking an average of mulitple readings to more clearly see the changes in magnetic field.

Show Code for averaged hall sensor values

  int h;
  double hallAverage;
  void loop() {
    // Take an average of 1000 readings over a period of 100 milliseconds.
    h = 0;
    for (int i=0;i<1000;i++) {
      h += hallRead();
      delayMicroseconds(100);
    }
    hallAverage = (double) h / 1000.0;
  }

Making HTTP requests from ESP32.

Now, let's start sending these values to ThingSpeak with HTTP.

• Install the library ArduinoHttpClient from Library Manager, it makes writing HTTP requests more convenient.
• Import the WiFi and ArduinoHttpClient libraries (at the top of your Arduino code):
  #include <WiFi.h>
  #include <ArduinoHttpClient.h>
• At the beginning of the code, define these constants and update their values:

const char* ssid     = "SOME_WIFI_SSID";
const char* password = "SOME_WIFI_PASSWORD";

const char serverAddress[] = "api.thingspeak.com";
const char* apikey = "YOUR_THINGSPEAK_API_KEY_HERE";

• Initialize the WiFi connection when device boots with the following:
  • Show Code

WiFiClient wifi;

void setup() {
  Serial.begin(115200); 
  Serial.println("Attempting to connect to Network named: ");
  Serial.println(ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while ( WiFi.status() != WL_CONNECTED) {
    // Connect to WPA/WPA2 network: 
    Serial.print(".");
    delay(500);
  }

  // print the SSID of the network you're attached to:
  Serial.println("Connected to SSID: " + WiFi.SSID());
  Serial.print("IP Address: ");
  Serial.println( WiFi.localIP() );  
}

Making the request:

• Outside of loop() or setup(), initialize of the HttpClient:
  HttpClient client = HttpClient(wifi, serverAddress, 80);
• Then, read Hall sensor value and send it with HTTP GET with:
  • Show Code

int sensorValue = 0;

void loop() {
  sensorValue = hallRead(); // or replace with averaged value code from above
  Serial.println("Sending sensor value: " + String(sensorValue));

  String url = "/update";
  url += "?api_key=" + String(apikey);
  url += "&field1=" + String(sensorValue);

  client.get(url); // perform the GET request

  // read the status code and body of the response
  int statusCode = client.responseStatusCode();
  String response = client.responseBody();

  Serial.print("Status code: ");
  Serial.println(statusCode);
  Serial.print("Response: ");
  Serial.println(response);
  Serial.println("Wait 15 seconds");
  delay(15000);
}

If your ESP32 is successfully sending data, you should be able to see it in the visualization plot on ThingSpeak.

Part 2 - calculating aggregate statistics with MatLab.

An important feature of ThingSpeak is the capability of running MatLab Analysis. You can set up code which performs processing and/or decison-making and then writes new data to a channel or notifies an external web service. Some examples of what you could do:

• Combine data from multiple sources: your channel, other people's public channels and even external web services, do analysis based on tehm.
  • E.g. combine home air quality & heating data with a weather service and electricity grid price rates.
• If your end-device is sending sensor values as analog voltage values (as we have been doing with ESP32), convert it into a standard, human-interpretable value
  • E.g. converting the resistance read by Arduino from a thermistor into a value in Celsius.

The functions using which you can read/write to Channels in Thingspeak Matlab are:

• thingSpeakRead( .. )
• thingSpeakWrite( .. )

They support various parameters, such as whether to fetch data based on a time-window or number of measurements.

Let's create a simple average sensor value calculation. In Thingspeak web interface:

1. Apps -> MATLAB Analysis. Create a new Analysis.
2. Choose the example "Calculate and display average humidity".
   • Modify it, so instead of humidity, we use our field1- magnetic value
   • Instead of reading data for last 60 minutes, try to read last 10 values:
     'NumPoints',10 instead of 'NumMinutes', 60
   • Try to run the analysis, don't use thingSpeakWrite() for now. You should see the average value in the "Output" panel below. If you see any errors instead, try to fix them.

• Create a 2nd channel, we will write the analysis results here. Add 3 fields to it - min, max, mean.
  • Add calculation of min and max.
  • Write the data to our new channel - study the documentation of ThingSpeakWrite. We need to provide which fields we want to update. For example, if we want to update fields 5-7, the code would look like below.
    • thingSpeakWrite(CHANNEL_ID,'Fields',[5,6,7],'Values',{2.3,'on','good'},'WriteKey','CHANNEL_API_KEY')
  • Adapt the code so that you write the min, max and mean to the 2nd channel.

Running the analysis periodically with TimeControl

• Finally, to make the MatLab analysis run at a time interval, you have to create a TimeControl app (Apps -> TimeControl).
  • Create a recurring TimeControl which triggers the Matlab code every 5 minutes.

Part 3 - Sending commands to the ESP32

ThingSpeak has a module called TalkBack. Clients can add new commands to a (sorted) queue and end-devices can fetch new commands to execute from the queue with a polling mechanism. We will create commands to turn the ESP32 integrated LED on or off.

• Note that the TalkBack key is different than the channel key we used before!

Create a new TalkBack module. This will produce another API key and TalkBack module ID.

Let's replace the Arduino loop() code so that instead of sending sensor values, we are polling for new commands. Let's define two commands: "TURN_ON" and "TURN_OFF", and make the ESP32 respectively turn the in-built blue LED on/off.

Add your API key and ID as constants. Also define the pin of the integrated LED:

 
unsigned long myTalkBackID = 12345;
const char* myTalkBackKey = "XXXXXX";
const int LED_BUILTIN = 2;

• Modify the loop() code by replacing the request with:
  • Talkback Request Show Code

• Now, check the response code and response value, whether it contains a command:
  • Show Code for TalkBack response handling

// Check the result
if (statusCode == 200) {
  Serial.println("Checking TalkBack queue for new commands..."); 

  if(response.length() != 0) {
    Serial.print("  Latest command from queue: ");
    Serial.println(response);

    if(response == "TURN_ON") {
	  digitalWrite(LED_BUILTIN, HIGH);  
    } if(response == "TURN_OFF") {
	  digitalWrite(LED_BUILTIN, LOW);
    }

  } else {
    Serial.println("  No new commands.");  
  }

} else {
  Serial.println("Problem checking queue. HTTP status code " + String(statusCode));
}

Test it out by manually adding new commands from the TalkBack web app.

Part 4 - Feedback loop - sending data & reacting to it.

Now, let's update the ESP32 to both send data and listen for new commands. Secondly, we want ThingSpeak to automatically add a new command if the sensor values are below a threshold.

First, let's write the Arduino code. We could make two separate requests (one for writing data, one for fetching new commands). But ThingSpeak API also supports a request to do both at once.

• Try to update the POST request from part 3 so that it uses the above combined POST request instead and also sends the hall sensor value.

Data-based triggers in ThingSpeak with React

To complete the loop, let's make Thingspeak react on the data we are sending. We want a new command to be added to the TalkBack queue automatically, if the sensor value is above a certain limit.

We will use additional apps from ThingSpeak to achieve this:

1. React - set up some rules on data values (e.g. a treshold) which trigger some behaviour (e.g. send a tweet)
2. ThingHTTP - an app for defining HTTP requests that ThingSpeak can perform. A ThingHTTP request can be invoked by React or TimeControl.

React has the limitation - it can't directly add new TalkBack commands. However, what it can do is invoke ThingHTTP.

Meanwhile, the TalkBack REST API supports adding new commands via HTTP. So, we can define a ThingHTTP request which creates a new TalkBack Command. Then, we can set up a React rule to invoke that ThingHTTP request based on our data.

The flow will be:

ESP32 -> ThingSpeak Channel -> React -> ThingHTTP -> TalkBack API -> ESP32

Note: it's also possible to make HTTP requests from ThingSpeak Matlab code using webwrite.

• Create a new ThingHTTP module which invokes the TalkBack API:

• Create a React module, which triggers the ThingHTTP if the hall sensor value exceeds some limit (based on the values you have been getting from the sensor), e.g when it is below 0.
  • For condition type use "Numeric"

Verify that the loop works.

Other information

There is also an Arduino ESP32/ESP8266 Thingspeak Library, if you prefer to avoid writing the HTTP requests yourself as we did in this lab.