Summary

In part7 of the Tilt Hydrometer series I will create a new task called the “Tilt Data Manager” which will manage the current state of all of the Tilt Hydrometers.

Story

In the design of my system I want the Bluetooth Manager to get a high priority so that it can react to all of the advertising reports without dropping data.  In addition I want other tasks in the system to be able to find out about the state of “Tilts” without distracting the Bluetooth subsystem from its job, collecting data.  To that end I will create a new task called the “Tilt Data Manager” which have a command queue where the Bluetooth Manager can submit advertising reports and other tasks can ask for data. For example the future Display Manager task can ask for the current state of the Pink Tilt.  In addition this task will filter down the blast of data to a more reasonable amount.  In the picture below, you can see the current state of the architecture in blue, and the new updates in green.

 

This series is broken up into the following 12 articles with a few additional possible articles. 

Tilt Hydrometer (Part 1) Overview & Out-of-Box

Tilt Hydrometer (Part 2) Architecture

Tilt Hydrometer (Part 3) Advertising Scanner

Tilt Hydrometer (Part 4) Advertising Packet Error?

Tilt Hydrometer (Part 5) Tilt Simulator & Multi Advertising iBeacons

Tilt Hydrometer (Part 6) Tilt Simulator an Upgrade

Tilt Hydrometer (Part 7) Advertising Database

Tilt Hydrometer (Part 8) Read the Database

Tilt Hydrometer (Part 9) An LCD Display

Tilt Hydrometer (Part 10) The Display State Machine

Tilt Hydrometer (Part 11) Draw the Display Screens

Tilt Hydrometer (Part 12) CapSense

Tilt Hydrometer: LittleFS & SPI Flash (Part ?)

Tilt Hydrometer: WiFi Introducer (Part ?)

Tilt Hydrometer: WebServer (Part ?)

Tilt Hydrometer: Amazon MQTT (Part ?)

Tilt Hydrometer: Printed Circuit Board (Part ?)

You can get the source code from git@github.com:iotexpert/Tilt2.git  This repository has tags for each of the articles which can be accessed with "git checkout part12"  You can find the Tilt Simulator at  git@github.com:iotexpert/TiltSimulator.git.

 

Tilt Database

The first step is to build the data structures that will represent the database.  The question is, what do I want to save and how do I want to save it?  The answer to the first question is:

  1. Gravity
  2. Temperature
  3. RSSI
  4. TxPower
  5. Time (when the report was made)

The answer to how is, a “struct” that represents one data point with the current data, and a pointer which can be used to make a linked list.  In future articles I will send this data structure around to the other tasks (like the display or filesystem) so I will put this structure into the public interface file “tiltDataManager.h”

#pragma once

#include <stdint.h>
#include "FreeRTOS.h"
#include "queue.h"
#include "wiced_bt_ble.h"

typedef struct {
    float gravity;
    int temperature;
    int8_t rssi;
    int8_t txPower;
	uint32_t time;
    struct tdm_tiltData_t *next;
} tdm_tiltData_t;

You might recall from the previous article that I have an array of structures that hold information about the Tilts, one for Red, Green, Black, etc. I decided that rather than other tasks asking for a “int” that they should ask for a handle which is an alias for that int.  So, I define that next.

typedef int tdm_tiltHandle_t;

The next step in building the database is to yank the struct and array out of the BluetoothManager.c and put it where it belongs, in the tiltDataManager.c.  In the list of devices I add a pointer to the “data” which is just a list of the datapoint defined in the structure tdb_tiltData_t.  In addition I add “numDataPoints” because I am going to implement a running average.  And a “numDataSeen” which is the total number that I have ever seen.  Here is the data structure:

#define TILT_IBEACON_HEADER_LEN 20
#define TILT_IBEACON_DATA_LEN 5
#define TILT_IBEACON_LEN (TILT_IBEACON_HEADER_LEN + TILT_IBEACON_DATA_LEN)
typedef struct  {
    char *colorName;
    uint8_t uuid[TILT_IBEACON_HEADER_LEN];
    tdm_tiltData_t *data;
    int numDataPoints;
    int numDataSeen;
} tilt_t;

#define IBEACON_HEADER 0x4C,0x00,0x02,0x15

static tilt_t tiltDB [] =
{
    {"Red",      {IBEACON_HEADER,0xA4,0x95,0xBB,0x10,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Green" ,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x20,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Black" ,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x30,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Purple",   {IBEACON_HEADER,0xA4,0x95,0xBB,0x40,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Orange",   {IBEACON_HEADER,0xA4,0x95,0xBB,0x50,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Blue"  ,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x60,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Yellow",   {IBEACON_HEADER,0xA4,0x95,0xBB,0x70,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Pink"  ,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x80,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
};
#define NUM_TILT (sizeof(tiltDB)/sizeof(tilt_t))

Update the Public Interface to the tiltDataManager

Back in the tiltDataManager.h I will add two public function definitions

  1. The task (so that it can be started in main)
  2. A function tdm_processIbeacon which the BluetoothManager can call to consume a datapoint
void tdm_task(void *arg);
void tdm_processIbeacon(uint8_t *mfgAdvField,int len,wiced_bt_ble_scan_results_t *p_scan_result);

Modify the Bluetooth Manager to Submit

The next step is to yank out the old scanning callback in the Bluetooth Manager and replace it with a call to the new public interface to the tiltDataManager.  I spent a good bit of time thinking about where exactly I wanted the partition to exist between the Bluetooth Manager and the Tilt Data Manager.  I decided that the Tilt Data Manager would need to know how to decode the advertising packet, but only in that one function.

case BTM_ENABLED_EVT:
            if (WICED_BT_SUCCESS == p_event_data->enabled.status)
            {
				wiced_bt_ble_observe(WICED_TRUE, 0,tdm_processIbeacon);
            }

Build the Command Queue

The Tilt Data Manager will just “sit on a queue”, specifically a command queue.  When it receives messages from various sources it will do the needful.  For now there are two messages, TDM_CMD_ADD_DATA_POINT and TDM_CMD_PROCESS_DATA.  The message is a structure with

  1. What is the command
  2. Which Tilt are we talking about
  3. A generic void pointer
typedef enum {
    TDM_CMD_ADD_DATAPOINT,
    TDM_CMD_PROCESS_DATA,
} tdm_cmd_t;

typedef struct {
    tdm_cmd_t cmd;
    tdm_tiltHandle_t tilt;
    void *msg;
} tdm_cmdMsg_t;

static QueueHandle_t tdm_cmdQueue;

Process the Command Queue & tdm_task

The main Tilt Data Manager task has two things going on

  1. A queue where messages can be sent
  2. A periodic timer which will insert the message “TDM_CMD_PROCESS_DATA”

The main tasks just waits for messages to be pushed into the command queue.

void tdm_task(void *arg)
{
    tdm_cmdQueue = xQueueCreate(10,sizeof(tdm_cmdMsg_t));
    tdm_cmdMsg_t msg;

    TimerHandle_t tdm_processDataTimer;

    // Call the process data function once per hour to filter the input data
    tdm_processDataTimer=xTimerCreate("process",1000*30,pdTRUE,0,tdm_submitProcessData);
    xTimerStart(tdm_processDataTimer,0);

    while(1)
    {
        xQueueReceive(tdm_cmdQueue,&msg,portMAX_DELAY);
        switch(msg.cmd)
        {
            case TDM_CMD_ADD_DATAPOINT:
                tdm_addData(msg.tilt,msg.msg);
            break;

            case TDM_CMD_PROCESS_DATA:
                tdm_processData();
            break;
        }
    }
}

Process the Data

Every 30 seconds a Process Data message is pushed into the command queue.  When that happens it calls the function tdm_processData which will:

  1. Iterate through the list of tilts
  2. If there is data
  3. Calculate the running average
  4. Print out the current value
// There is a bunch of data coming out of the tilts... every second on 3 channels
// So you may endup with a boatload of data
// This function will take an average of all of the data that has been saved

static void tdm_processData()
{
    for(int i=0;i<NUM_TILT;i++)
    {
        if(tiltDB[i].data == 0)
        {
            continue;
        }

        tiltDB[i].data->gravity /= tiltDB[i].numDataPoints;
        tiltDB[i].data->temperature /= tiltDB[i].numDataPoints;
        tiltDB[i].numDataPoints = 1;
        printf("Tilt %s Temperature = %d Gravity =%f\n",tiltDB[i].colorName,tiltDB[i].data->temperature,tiltDB[i].data->gravity);
    }
}

Add Data

The add data function is called when a data point is submitted into the command queue.  When this happens there are two possible scenarios

  1. There is no history, in which case just setup the data pointer to this message
  2. There is history, in which case just add to the running total.
static void tdm_addData(tdm_tiltHandle_t handle, tdm_tiltData_t *data)
{
    if(tiltDB[handle].data == 0)
    {
        tiltDB[handle].data = data; 
        tiltDB[handle].numDataPoints = 1;
        tiltDB[handle].data->next=0;
    }
    else
    {
        tiltDB[handle].data->gravity     += data->gravity;
        tiltDB[handle].data->temperature += data->temperature;
        tiltDB[handle].numDataPoints += 1;

        free(data);
    }

    tiltDB[handle].numDataSeen += 1;
    tiltDB[handle].data->time    = data->time;
    tiltDB[handle].data->rssi    = data->rssi;
    tiltDB[handle].data->txPower = data->txPower;
}

Process the iBeacon

The process iBeacon function is actually called in the context of the Bluetooth Manager.  But, if that is true, why did I put it into the Tilt Data Manager file?  Answer: because it needs to know all about what Tilts look like.  This function is essentially the same as the one from the previous article.  It looks for an iBeacon in the manufacturer specific data.  If it find it, then it decodes the Gravity, Temperature and txPower.  Then puts that data into a structure and submits it to the Tilt Data Manager command queue.

//
// This function is called in the Bluetooth Manager Context
// Specifically it is the advertising observer callback
//
void tdm_processIbeacon(wiced_bt_ble_scan_results_t *p_scan_result,uint8_t *p_adv_data)
{

	uint8_t mfgFieldLen;
	uint8_t *mfgAdvField;
	mfgAdvField = wiced_bt_ble_check_advertising_data(p_adv_data,BTM_BLE_ADVERT_TYPE_MANUFACTURER,&mfgFieldLen);
    
	if(!mfgAdvField)
        return;

    if(mfgFieldLen != TILT_IBEACON_LEN)
        return;

    for(int i=0;i<NUM_TILT;i++)
    {
        if(memcmp(mfgAdvField,tiltDB[i].uuid,TILT_IBEACON_HEADER_LEN) == 0)
        {
            uint32_t timerTime = xTaskGetTickCount() / 1000;
		    int8_t txPower = mfgAdvField[24];
		    float gravity = ((float)((uint16_t)mfgAdvField[22] << 8 | (uint16_t)mfgAdvField[23]))/1000;
		    int temperature = mfgAdvField[20] << 8 | mfgAdvField[21];

            // The tilt repeater will send out 0's if it hasnt heard anything ... and they send out some crazy stuff
            // when they first startup 
            if(gravity>2.0 || gravity<0.95 || temperature > 110 || gravity == 0 || temperature == 0)
                return;

            tdm_tiltData_t *data;
            data = malloc(sizeof(tdm_tiltData_t));
                
            data->gravity     = gravity;
            data->temperature = temperature;
            data->txPower     = txPower;
            data->time        = timerTime;
            data->rssi        = p_scan_result->rssi;

            tdm_submitNewData(i,data);
            
        }
    }
}

Submit Commands

To simply submitting commands, I create two helper functions:

void tdm_submitNewData(tdm_tiltHandle_t handle,tdm_tiltData_t *data)
{
    tdm_cmdMsg_t msg;
    msg.msg = data;
    msg.tilt = handle;
    msg.cmd =  TDM_CMD_ADD_DATAPOINT;
    if(xQueueSend(tdm_cmdQueue,&msg,0) != pdTRUE)
    {
        printf("Failed to send to dmQueue\n");
        free(data);
    }
}

static void tdm_submitProcessData()
{
    tdm_cmdMsg_t msg;
    msg.cmd = TDM_CMD_PROCESS_DATA;
    xQueueSend(tdm_cmdQueue,&msg,0);
}

Update main.c

In main.c I need to start the tdm_task

    xTaskCreate(tdm_task, "Tilt Data Manager", configMINIMAL_STACK_SIZE*2,0 /* args */ ,0 /* priority */, 0);

Program and Test

Now lets test this thing and see if I have it working.  Now I am glad that I did the work to build a Tilt Simulator.  You can see the window on top is the simulator.  I tell it to start broadcasting “Red” with Temperature of 70, Gravity of 1010 and TxPower of 99.  Then on the bottom window you can see that is what is coming out of the system every 30 seconds.

In the next article I will add the LCD display.

tiltDataManager.h

#pragma once

#include <stdint.h>
#include "FreeRTOS.h"
#include "queue.h"
#include "wiced_bt_ble.h"

typedef struct {
    float gravity;
    int temperature;
    int8_t rssi;
    int8_t txPower;
	uint32_t time;
    struct tdm_tiltData_t *next;
} tdm_tiltData_t;

typedef int tdm_tiltHandle_t;

void tdm_task(void *arg);
void tdm_processIbeacon(wiced_bt_ble_scan_results_t *p_scan_result,uint8_t *p_adv_data);

tiltDataManager.c

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "wiced_bt_ble.h"
#include "FreeRTOS.h"
#include "task.h"
#include "queue.h"
#include "timers.h"
#include "tiltDataManager.h"
/*********************************************************************************
*
* Tilt Database Definition
* 
*********************************************************************************/
#define TILT_IBEACON_HEADER_LEN 20
#define TILT_IBEACON_DATA_LEN 5
#define TILT_IBEACON_LEN (TILT_IBEACON_HEADER_LEN + TILT_IBEACON_DATA_LEN)
typedef struct  {
char *colorName;
uint8_t uuid[TILT_IBEACON_HEADER_LEN];
tdm_tiltData_t *data;
int numDataPoints;
int numDataSeen;
} tilt_t;
#define IBEACON_HEADER 0x4C,0x00,0x02,0x15
static tilt_t tiltDB [] =
{
{"Red",      {IBEACON_HEADER,0xA4,0x95,0xBB,0x10,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
{"Green" ,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x20,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
{"Black" ,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x30,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
{"Purple",   {IBEACON_HEADER,0xA4,0x95,0xBB,0x40,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
{"Orange",   {IBEACON_HEADER,0xA4,0x95,0xBB,0x50,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
{"Blue"  ,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x60,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
{"Yellow",   {IBEACON_HEADER,0xA4,0x95,0xBB,0x70,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
{"Pink"  ,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x80,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
};
#define NUM_TILT (sizeof(tiltDB)/sizeof(tilt_t))
/*********************************************************************************
* 
* Tilt Data Manager External Interface Queue
*
*********************************************************************************/
typedef enum {
TDM_CMD_ADD_DATAPOINT,
TDM_CMD_PROCESS_DATA,
} tdm_cmd_t;
typedef struct {
tdm_cmd_t cmd;
tdm_tiltHandle_t tilt;
void *msg;
} tdm_cmdMsg_t;
static QueueHandle_t tdm_cmdQueue;
static void tdm_submitProcessData();
// There is a bunch of data coming out of the tilts... every second on 3 channels
// So you may endup with a boatload of data
// This function will take an average of all of the data that has been saved
static void tdm_processData()
{
for(int i=0;i<NUM_TILT;i++)
{
if(tiltDB[i].data == 0)
{
continue;
}
tiltDB[i].data->gravity /= tiltDB[i].numDataPoints;
tiltDB[i].data->temperature /= tiltDB[i].numDataPoints;
tiltDB[i].numDataPoints = 1;
printf("Tilt %s Temperature = %d Gravity =%f\n",tiltDB[i].colorName,tiltDB[i].data->temperature,tiltDB[i].data->gravity);
}
}
// This function will
// insert new data for that tilt if none has ever been seen
// or it will add the data to the current count
static void tdm_addData(tdm_tiltHandle_t handle, tdm_tiltData_t *data)
{
if(tiltDB[handle].data == 0)
{
tiltDB[handle].data = data; 
tiltDB[handle].numDataPoints = 1;
tiltDB[handle].data->next=0;
}
else
{
tiltDB[handle].data->gravity     += data->gravity;
tiltDB[handle].data->temperature += data->temperature;
tiltDB[handle].numDataPoints += 1;
free(data);
}
tiltDB[handle].numDataSeen += 1;
tiltDB[handle].data->time    = data->time;
tiltDB[handle].data->rssi    = data->rssi;
tiltDB[handle].data->txPower = data->txPower;
}
////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// Public Functions
// 
////////////////////////////////////////////////////////////////////////////////////////////////////////
void tdm_task(void *arg)
{
tdm_cmdQueue = xQueueCreate(10,sizeof(tdm_cmdMsg_t));
tdm_cmdMsg_t msg;
TimerHandle_t tdm_processDataTimer;
// Call the process data function once per hour to filter the input data
tdm_processDataTimer=xTimerCreate("process",1000*30,pdTRUE,0,tdm_submitProcessData);
xTimerStart(tdm_processDataTimer,0);
while(1)
{
xQueueReceive(tdm_cmdQueue,&msg,portMAX_DELAY);
switch(msg.cmd)
{
case TDM_CMD_ADD_DATAPOINT:
tdm_addData(msg.tilt,msg.msg);
break;
case TDM_CMD_PROCESS_DATA:
tdm_processData();
break;
}
}
}
////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// These functions submit commands to main command queue: tdm_cmdQueue
// 
////////////////////////////////////////////////////////////////////////////////////////////////////////
void tdm_submitNewData(tdm_tiltHandle_t handle,tdm_tiltData_t *data)
{
tdm_cmdMsg_t msg;
msg.msg = data;
msg.tilt = handle;
msg.cmd =  TDM_CMD_ADD_DATAPOINT;
if(xQueueSend(tdm_cmdQueue,&msg,0) != pdTRUE)
{
printf("Failed to send to dmQueue\n");
free(data);
}
}
static void tdm_submitProcessData()
{
tdm_cmdMsg_t msg;
msg.cmd = TDM_CMD_PROCESS_DATA;
xQueueSend(tdm_cmdQueue,&msg,0);
}
//
// This function is called in the Bluetooth Manager Context
// Specifically it is the advertising observer callback
//
void tdm_processIbeacon(wiced_bt_ble_scan_results_t *p_scan_result,uint8_t *p_adv_data)
{
uint8_t mfgFieldLen;
uint8_t *mfgAdvField;
mfgAdvField = wiced_bt_ble_check_advertising_data(p_adv_data,BTM_BLE_ADVERT_TYPE_MANUFACTURER,&mfgFieldLen);
if(!mfgAdvField)
return;
if(mfgFieldLen != TILT_IBEACON_LEN)
return;
for(int i=0;i<NUM_TILT;i++)
{
if(memcmp(mfgAdvField,tiltDB[i].uuid,TILT_IBEACON_HEADER_LEN) == 0)
{
uint32_t timerTime = xTaskGetTickCount() / 1000;
int8_t txPower = mfgAdvField[24];
float gravity = ((float)((uint16_t)mfgAdvField[22] << 8 | (uint16_t)mfgAdvField[23]))/1000;
int temperature = mfgAdvField[20] << 8 | mfgAdvField[21];
// The tilt repeater will send out 0's if it hasnt heard anything ... and they send out some crazy stuff
// when they first startup 
if(gravity>2.0 || gravity<0.95 || temperature > 110 || gravity == 0 || temperature == 0)
return;
tdm_tiltData_t *data;
data = malloc(sizeof(tdm_tiltData_t));
data->gravity     = gravity;
data->temperature = temperature;
data->txPower     = txPower;
data->time        = timerTime;
data->rssi        = p_scan_result->rssi;
tdm_submitNewData(i,data);
}
}
}

 

I am an engineer and tinkerer at heart. I live in the most beautiful place on earth - Central Kentucky. This blog is about hooking stuff up to the internet. Click here for the long version.

Recommended Posts

Tilt Hydrometer (Part 12) CapSense
Tilt Hydrometer (Part 11) Draw the Display Screens
Tilt Hydrometer (Part 10) The Display State Machine
Tilt Hydrometer (Part 9) An LCD Display

No comment yet, add your voice below!

Add a Comment

Your email address will not be published. Required fields are marked *