AnyCloud WiFi Template + WiFi Helper Library (Part 3): A New Modus Toolbox Library

Summary

Instructions to create a new Modus Toolbox / AnyCloud library including modifying your master middleware manifest and updating the dependencies.  The new library and dependencies will then be available in your library browser and new project creator.

Article
(Part 1) Create Basic Project & Add Cypress Logging Functionality
(Part 2) Create New Thread to manage WiFi using the Wireless Connection Manager
(Part 3) Create a New Middleware Library with WiFi helper functions
(Part 4) Add WiFi Scan
Add WiFi Connect
Add WiFi Disconnect
Add WiFi Ping
Add Gethostbyname
Add MDNS
Add Status
Add StartAP
Make a new template project (update manifest)

Story

In the previous article we discussed the steps to turn on the WiFi chip in your project using the Wireless Connection Manager Anycloud (WCM) library.  When something happens with the WCM it will give you a callback to tell you what happened.  In my example code there were three printf’s that were commented out for the conditions:

  • CY_WCM_EVENT_IP_CHANGED
  • CY_WCM_EVENT_STA_JOINED_SOFTAP
  • CY_WCM_EVENT_STA_LEFT_SOFTAP

The question you might have is “What is the new Ip Address”” or “What is the MAC address of the Station which joined the SoftAp?”

        case CY_WCM_EVENT_IP_CHANGED:           /**< IP address change event. This event is notified after connection, re-connection, and IP address change due to DHCP renewal. */
 //               cy_wcm_get_ip_addr(wifi_network_mode, &ip_addr, 1);
                printf("Station IP Address Changed: %s\n",wifi_ntoa(&ip_addr));
        break;
        case CY_WCM_EVENT_STA_JOINED_SOFTAP:    /**< An STA device connected to SoftAP. */
//            printf("STA Joined: %s\n",wifi_mac_to_string(event_data->sta_mac));
        break;
        case CY_WCM_EVENT_STA_LEFT_SOFTAP:      /**< An STA device disconnected from SoftAP. */
//            printf("STA Left: %s\n",wifi_mac_to_string(event_data->sta_mac));

So I wrote “standard” functions to

  • Convert an IP address structure to a string (like ntoa in Linux)
  • Convert a MAC address to a string

I essentially got these from the code example where they were redundantly repeatedly repeated.  After tweaking them to suit my liking I wanted to put them in a library.

Make the C-Library

Follow these steps to make the c-library.  First, make a new directory in your project called “wifi_helper”.  You can do this in Visual Studio Code by pressing the folder button with the plus on it.

Then create the files wifi_helper.h and wifi_helper.c

In “wifi_helper.h” type in the public interface.  Specifically, that we want a function that takes a mac address returns a char*.  And another function that takes an IP address and returns a char*

#pragma once

#include "cy_wcm.h"

char *wifi_mac_to_string(cy_wcm_mac_t mac);

char *wifi_ntoa(cy_wcm_ip_address_t *ip_addr);


All right Hassane… yes these functions need comments.  Notice that I allocated a static buffer inside of these two function.  That means that these functions are NOT NOT NOT thread safe.  However, personally I think that is fine as I think that it is unlikely that they would ever be called from multiple threads.

#include "wifi_helper.h"
#include "cy_wcm.h"
#include <stdio.h>
#include "cy_utils.h"
#include "cy_log.h"

char *wifi_mac_to_string(cy_wcm_mac_t mac)
{
    static char _mac_string[] = "xx:xx:xx:xx:xx:xx";
    sprintf(_mac_string,"%02X:%02X:%02X:%02X:%02X:%02X",mac[0],mac[1],mac[2],mac[3],mac[4],mac[5]);
    return _mac_string; 
}


char *wifi_ntoa(cy_wcm_ip_address_t *ip_addr)
{
    static char _netchar[32];
    switch(ip_addr->version)
    {
        case CY_WCM_IP_VER_V4:
            sprintf(_netchar,"%d.%d.%d.%d", (uint8_t)ip_addr->ip.v4,
                (uint8_t)(ip_addr->ip.v4 >> 8), (uint8_t)(ip_addr->ip.v4 >> 16),
                (uint8_t)(ip_addr->ip.v4 >> 24));        break;
        case CY_WCM_IP_VER_V6:
            sprintf(_netchar,"%X:%X:%X:%X", (uint8_t)ip_addr->ip.v6[0],
                (uint8_t)(ip_addr->ip.v6[1]), (uint8_t)(ip_addr->ip.v6[2]),
                (uint8_t)(ip_addr->ip.v6[3]));
        break;
    }
    CY_ASSERT(buff[0] != 0); // SOMETHING should have happened
    return _netchar;
}

Git Repository

Now that I have the files I need in the library, I want to create a place on GitHub to hold the library.

Now we need to integrate the files into Git.  To do this you need to

  1. Initialize a new git repository (git init .)
  2. Add a remote (git remote add origin git@github.com:iotexpert/wifi_helper.git)
  3. Pull the remote files (README and LICENSE) with (git pull origin main)
  4. Add the wifi_helper files (git add wifi_helper.*)
  5. Commit the changes (git commit -m “added initial c files”)
  6. Push them to the remote (git push -u origin main)
arh (master *+) wifi_helper $ pwd
/Users/arh/proj/elkhorncreek3/IoTExpertWiFiTemplate/wifi_helper
arh (master *+) wifi_helper $ git init .
Initialized empty Git repository in /Users/arh/proj/elkhorncreek3/IoTExpertWiFiTemplate/wifi_helper/.git/
arh (main #) wifi_helper $ git remote add origin git@github.com:iotexpert/wifi_helper.git
arh (main #) wifi_helper $ git pull origin main
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (4/4), 1.28 KiB | 436.00 KiB/s, done.
From iotexpert.github.com:iotexpert/wifi_helper
 * branch            main       -> FETCH_HEAD
 * [new branch]      main       -> origin/main
arh (main) wifi_helper $ git add wifi_helper.*
arh (main +) wifi_helper $ git commit -m "added initial c files"
[main f7d10b1] added initial c files
 2 files changed, 72 insertions(+)
 create mode 100644 wifi_helper.c
 create mode 100644 wifi_helper.h
arh (main) wifi_helper $ git push -u origin main
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 12 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 1.10 KiB | 1.10 MiB/s, done.
Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
To iotexpert.github.com:iotexpert/wifi_helper.git
   3a1ad32..f7d10b1  main -> main
Branch 'main' set up to track remote branch 'main' from 'origin'.
arh (main) wifi_helper $ 

Now you will have something like this on GitHub.

Manifest Files

I would like to be able to have my new library show up in the library browser.  But how?  When the library browser starts up it needs to discover:

  1. Board Support Packages
  2. Template Projects
  3. Middleware Code Libraries

To do this, it reads a series of XML files called “manifests”.  These manifest files tell the library browser where to find the libraries.  If you have ever noticed the library browser (or the new project creator) it looks like this:

The message “Processing super-manifest …” give you a hint to go to https://raw.githubusercontent.com/cypresssemiconductorco/mtb-super-manifest/v2.X/mtb-super-manifest-fv2.xml

Here it is.  Notice that the XML scheme says that this file is a “super-manifest”.  Then notice that there are sections:

  • <board-manifest-list> these are BSPs
  • <app-manifest-list> these are template projects
  • <middleware-manifest-list> these are middleware code libraries
<super-manifest>
  <board-manifest-list>
    <board-manifest>
      <uri>https://github.com/cypresssemiconductorco/mtb-bsp-manifest/raw/v2.X/mtb-bsp-manifest.xml</uri>
    </board-manifest>
    <board-manifest dependency-url="https://github.com/cypresssemiconductorco/mtb-bsp-manifest/raw/v2.X/mtb-bsp-dependencies-manifest.xml">
      <uri>https://github.com/cypresssemiconductorco/mtb-bsp-manifest/raw/v2.X/mtb-bsp-manifest-fv2.xml</uri>
    </board-manifest>
    <board-manifest>
      <uri>https://github.com/cypresssemiconductorco/mtb-bt-bsp-manifest/raw/v2.X/mtb-bt-bsp-manifest.xml</uri>
    </board-manifest>
    <board-manifest dependency-url="https://github.com/cypresssemiconductorco/mtb-bt-bsp-manifest/raw/v2.X/mtb-bt-bsp-dependencies-manifest.xml">
      <uri>https://github.com/cypresssemiconductorco/mtb-bt-bsp-manifest/raw/v2.X/mtb-bt-bsp-manifest-fv2.xml</uri>
    </board-manifest>
  </board-manifest-list>
  <app-manifest-list>
    <app-manifest>
      <uri>https://github.com/cypresssemiconductorco/mtb-ce-manifest/raw/v2.X/mtb-ce-manifest.xml</uri>
    </app-manifest>
    <app-manifest>
      <uri>https://github.com/cypresssemiconductorco/mtb-ce-manifest/raw/v2.X/mtb-ce-manifest-fv2.xml</uri>
    </app-manifest>
    <app-manifest>
      <uri>https://github.com/cypresssemiconductorco/mtb-bt-app-manifest/raw/v2.X/mtb-bt-app-manifest.xml</uri>
    </app-manifest>
    <app-manifest>
      <uri>https://github.com/cypresssemiconductorco/mtb-bt-app-manifest/raw/v2.X/mtb-bt-app-manifest-fv2.xml</uri>
    </app-manifest>
  </app-manifest-list>
  <middleware-manifest-list>
    <middleware-manifest>
      <uri>https://github.com/cypresssemiconductorco/mtb-mw-manifest/raw/v2.X/mtb-mw-manifest.xml</uri>
    </middleware-manifest>
    <middleware-manifest dependency-url="https://github.com/cypresssemiconductorco/mtb-mw-manifest/raw/v2.X/mtb-mw-dependencies-manifest.xml">
      <uri>https://github.com/cypresssemiconductorco/mtb-mw-manifest/raw/v2.X/mtb-mw-manifest-fv2.xml</uri>
    </middleware-manifest>
    <middleware-manifest>
      <uri>https://github.com/cypresssemiconductorco/mtb-bt-mw-manifest/raw/v2.X/mtb-bt-mw-manifest.xml</uri>
    </middleware-manifest>
    <middleware-manifest dependency-url="https://github.com/cypresssemiconductorco/mtb-bt-mw-manifest/raw/v2.X/mtb-bt-mw-dependencies-manifest.xml">
      <uri>https://github.com/cypresssemiconductorco/mtb-bt-mw-manifest/raw/v2.X/mtb-bt-mw-manifest-fv2.xml</uri>
    </middleware-manifest>
    <middleware-manifest>
      <uri>https://github.com/cypresssemiconductorco/mtb-wifi-mw-manifest/raw/v2.X/mtb-wifi-mw-manifest.xml</uri>
    </middleware-manifest>
    <middleware-manifest dependency-url="https://github.com/cypresssemiconductorco/mtb-wifi-mw-manifest/raw/v2.X/mtb-wifi-mw-dependencies-manifest.xml">
      <uri>https://github.com/cypresssemiconductorco/mtb-wifi-mw-manifest/raw/v2.X/mtb-wifi-mw-manifest-fv2.xml</uri>
    </middleware-manifest>
  </middleware-manifest-list>
</super-manifest>

But you can’t modify this to add your own?  So what do you do now?  Cypress put in the capability for you to extend the system by creating a file called “~/.modustoolbox/manifest.loc”.  This file contains one or more URLs to super-manifest files (like the one above) where you can add whatever you want.

Here is the iotexpert manifest.loc

arh ~ $ cd ~/.modustoolbox/
arh .modustoolbox $ more manifest.loc
https://github.com/iotexpert/mtb2-iotexpert-manifests/raw/master/iotexpert-super-manifest.xml
arh .modustoolbox $

This file points to a super manifest file in a GitHub repository.  Here is the repository:

Notice that it has

  • iotexpert-super-manifest.xml – the top level iotexpert manifest
  • iotexpert-app-manifest.xml – my template projects
  • iotexpert-mw-manifest.xml – my middleware
  • manifest.loc – the file you need to put in your home directory
  • iotexpert-mw-dependencies.xml – a new file which I will talk about later

And the super manifest file that looks like this:

<super-manifest>
  <board-manifest-list>
  </board-manifest-list>
  
  <app-manifest-list>
    <app-manifest>
      <uri>https://github.com/iotexpert/mtb2-iotexpert-manifests/raw/master/iotexpert-app-manifest.xml</uri>
      </app-manifest>
   </app-manifest-list>
  <board-manifest-list>
  </board-manifest-list>
  <middleware-manifest-list>
    <middleware-manifest dependency-url="https://github.com/iotexpert/mtb2-iotexpert-manifests/raw/master/iotexpert-mw-dependencies.xml">
      <uri>https://github.com/iotexpert/mtb2-iotexpert-manifests/raw/master/iotexpert-mw-manifest.xml</uri>
    </middleware-manifest>
  </middleware-manifest-list>
</super-manifest>

To add the library we created above, I need to add the new middleware into my middleware manifest.  Modify the file “iotexpert-mw-manifest.xml” to have the new middleware.

<middleware>
  <name>WiFi Helper Utilties</name>
  <id>wifi_helper</id>
  <uri>https://github.com/iotexpert/wifi_helper</uri>
  <desc>A library WiFi Helper utilities (e.g. aton)</desc>
  <category>IoT Expert</category>
  <req_capabilities>psoc6</req_capabilities>
  <versions>
    <version flow_version="2.0">
      <num>main</num>
      <commit>main</commit>
      <desc>main</desc>
    </version>
  </versions>
</middleware>

If you recall I have the “wifi_helper” directory inside of my project.  Not what I want (because I want it to be pulled using the library browser).  So I move out my project directory.  Now, let’s test the whole thing by running the library browser.

arh (master *+) IoTExpertWiFiTemplate $ pwd
/Users/arh/proj/elkhorncreek3/IoTExpertWiFiTemplate
arh (master *+) IoTExpertWiFiTemplate $ mv wifi_helper/ ~/proj/
arh (master *+) IoTExpertWiFiTemplate $ make modlibs
Tools Directory: /Applications/ModusToolbox/tools_2.3
CY8CKIT-062S2-43012.mk: ./libs/TARGET_CY8CKIT-062S2-43012/CY8CKIT-062S2-43012.mk
Launching library-manager

Excellent the WiFI Helper utilities show up.

And when I run the “update” the files show up in the project.

Add Dependencies

If you recall from the code I had this include:

#include "cy_wcm.h"

That means that I am dependent on the library “wifi-connection-manager”.  To make this work I create a new file called “iotexpert-mw-depenencies.xml”.  In that file I tell the system that “wifi_helper” is now dependent on “wcm”

<dependencies version="2.0">
  <depender>
    <id>wifi_helper</id>
    <versions>
      <version>
        <commit>main</commit>
        <dependees>
          <dependee>
            <id>wcm</id>
            <commit>latest-v2.X</commit>
          </dependee>
        </dependees>
      </version>
    </versions>
  </depender>
</dependencies>

Once I have that file, I add that depencency file to my middleware manifest file.

  <middleware-manifest-list>
    <middleware-manifest dependency-url="https://github.com/iotexpert/mtb2-iotexpert-manifests/raw/master/iotexpert-mw-dependencies.xml">
      <uri>https://github.com/iotexpert/mtb2-iotexpert-manifests/raw/master/iotexpert-mw-manifest.xml</uri>
    </middleware-manifest>
  </middleware-manifest-list>
</super-manifest>

Now when I start the library browser and add the “WiFi Help Utilities” it will automatically add the wireless connection manager (and all of the libraries that the wcm is dependent on.

In the next article I will add Scanning functionality to the WiFi Task.

AnyCloud WiFi Template + WiFi Helper Library (Part 2): Enable the WiFi Network

Summary

Instructions on using the AnyCloud Wireless Connection manager to enable WiFi.  This article is Part 2 of a series that will build a new IoT Expert template project for WiFi.

Article
(Part 1) Create Basic Project & Add Cypress Logging Functionality
(Part 2) Create New Thread to manage WiFi using the Wireless Connection Manager
(Part 3) Create a New Middleware Library with WiFi helper functions
(Part 4) Add WiFi Scan
Add WiFi Connect
Add WiFi Disconnect
Add WiFi Ping
Add Gethostbyname
Add MDNS
Add Status
Add StartAP
Make a new template project (update manifest)

Story

In the last article I got things going by starting from the old template, fixing up the Visual Studio Code configuration, adding the new Cypress Logging functionality and then testing everything.

In this article I will

  1. Create new task to manage WiFi
  2. Add Wireless Connection Manager to the project
  3. Create wifi_task.h and wifi_task.c
  4. Update usrcmd.c to send commands to the WiFi task

Create New Task to Manage WiFi

I am going to start by creating new a new task (called wifi_task) that will be responsible for managing the WiFi connection.  In Visual Studio Code you can create a new file by pressing the little document with the + on it.  You will need a file “wifi_task.h” and one “wifi_task.c”

Once you have wifi_task.h you will need to add the function prototype for the wifi_task.  In addition add a “guard”.  I like to use “#pragma once”

Here is copyable code.

#pragma once
void wifi_task(void *arg);

In wifi_task.c Ill start with a simple blinking LED function.  Well actually it will do a print instead of a blink.  Here it is:

#include "wifi_task.h"
#include "FreeRTOS.h"
#include "task.h"
#include <stdio.h>

void wifi_task(void *arg)
{
    while(1)
    {
        vTaskDelay(1000);
        printf("blink\n");

    }
}

Now that I have a wifi_task (which doesn’t do much) lets update main.c.  First include the wifi_task.h

#include "wifi_task.h"

Then create the task.  Notice that I start with a pretty big stack.

xTaskCreate(wifi_task,   "WiFi"       , configMINIMAL_STACK_SIZE*20,0 /* args */ ,0 /* priority */, 0);

When you run this, you will have the “blink” interleaved with the blink from the last article.

Add Wireless Connection Manager

You next step is to add the wireless connection manager.  Start the library browser by running “make modlibs”.  Then click on the wifi-connection-manager”.  Notice that when you do that, it will bring in a bunch of other libraries as well.  These are all libraries that it (the WiFi-Connection-Manager) is depend on.

If you look in the wireless connection manager documentation you will find this nice note.  It says that the WCM uses the Cypress logging functionality and you can turn it on with a #define.  That’s cool.  So now I edit the Makefile and add the define.

The the documentation also says that this library depends on the “Wi-Fi Middleware Core”

If you go to the Wi-Fi Middleware core library documentation you will see instructions that say that you need to

  1. Enable & Configure LWIP
  2. Enable & Configure MBEDTLS
  3. Enable & Configure the Cypress RTOS Abstraction

In order to do that you will need to two things

  1. Copy the configuration files into your project
  2. Setup some options in the Makefile

Start by copying the file.  They give you default configurations in mtb_share/wifi-mw-core/version/configs.  You will want to copy those files into your project.  This can be done in the Visual Studio Code interface using ctrl-c and ctrl-v

Notice that I now have two FreeRTOSConfig files.  So, delete the original file and rename the copied file.

Now your project should look like this:

The next step is to fix the Makefile by adding some defines.

 

DEFINES=CY_RETARGET_IO_CONVERT_LF_TO_CRLF
DEFINES+=CYBSP_WIFI_CAPABLE CY_RTOS_AWARE
DEFINES+=MBEDTLS_USER_CONFIG_FILE='"mbedtls_user_config.h"'
DEFINES+=ENABLE_WIFI_MIDDLEWARE_LOGS

The add the required components

COMPONENTS=FREERTOS LWIP MBEDTLS PSOC6HAL

Update wifi_task.c

My wifi task is going to work by

  1. Sitting on Queue waiting for messages of “wifi_cmd_t”
  2. When those messages come in, execute the right command.

Start by adding some includes to the wifi_task.c

#include <stdio.h>
#include "wifi_task.h"
#include "FreeRTOS.h"
#include "task.h"
#include "queue.h"

#include "cy_wcm.h"

Then define the legal commands.  I will add a bunch of more commands in the future articles.  But for this article there will be only one command.  Enable.  The command message is

  1. The command
  2. Three args of unknown type

In addition you will require

  1. The command queue
  2. An initialize state variable
  3. A way to keep track of what mode you are in (AP, STA or APSTA)
typedef enum {
    WIFI_CMD_ENABLE,

} wifi_cmd_t;

typedef struct {
    wifi_cmd_t cmd;
    void *arg0;
    void *arg1;
    void *arg2;
} wifi_cmdMsg_t;


static QueueHandle_t wifi_cmdQueue;
static bool wifi_initialized=false;
static cy_wcm_interface_t wifi_network_mode;

The first “command” that I will create is the enable.  This will

  1. Setup the interface
  2. Initialize the WiFi.  The simple init command actually does a bunch of stuff, including powering on the wifi chip, downloading the firmware into it, setting up all of the tasks in the RTOS, enabling the LWIP and MBEDTLS
static void wifi_enable(cy_wcm_interface_t interface)
{
    cy_rslt_t result;
    cy_wcm_config_t config = {.interface = interface}; 
    result = cy_wcm_init(&config); // Initialize the connection manager
    CY_ASSERT(result == CY_RSLT_SUCCESS);

    result = cy_wcm_register_event_callback(wifi_network_event_cb);
    CY_ASSERT(result == CY_RSLT_SUCCESS);

    wifi_network_mode = interface;
    wifi_initialized = true;
    
    printf("\nWi-Fi Connection Manager initialized\n");
    
}

In the previous block of code notice that I register a callback.  The callback looks like a switch that prints out messages based on the event type.  Notice that there are three lines which are commented out – which we will fix in the next article.

static void wifi_network_event_cb(cy_wcm_event_t event, cy_wcm_event_data_t *event_data)
{
    cy_wcm_ip_address_t ip_addr;

    switch(event)
    {
        case CY_WCM_EVENT_CONNECTING:            /**< STA connecting to an AP.         */
            printf("Connecting to AP ... \n");
        break;
        case CY_WCM_EVENT_CONNECTED:             /**< STA connected to the AP.         */
            printf("Connected to AP and network is up !! \n");
        break;
        case CY_WCM_EVENT_CONNECT_FAILED:        /**< STA connection to the AP failed. */
            printf("Connection to AP Failed ! \n");
        break;
        case CY_WCM_EVENT_RECONNECTED:          /**< STA reconnected to the AP.       */
            printf("Network is up again! \n");
        break;
        case CY_WCM_EVENT_DISCONNECTED:         /**< STA disconnected from the AP.    */
            printf("Network is down! \n");
        break;
        case CY_WCM_EVENT_IP_CHANGED:           /**< IP address change event. This event is notified after connection, re-connection, and IP address change due to DHCP renewal. */
                cy_wcm_get_ip_addr(wifi_network_mode, &ip_addr, 1);
   //             printf("Station IP Address Changed: %s\n",wifi_ntoa(&ip_addr));
        break;
        case CY_WCM_EVENT_STA_JOINED_SOFTAP:    /**< An STA device connected to SoftAP. */
 //           printf("STA Joined: %s\n",wifi_mac_to_string(event_data->sta_mac));
        break;
        case CY_WCM_EVENT_STA_LEFT_SOFTAP:      /**< An STA device disconnected from SoftAP. */
//            printf("STA Left: %s\n",wifi_mac_to_string(event_data->sta_mac));
        break;
    }

}

Now I want to update the main loop of the WiFI task.  It is just an infinite loop that processes command messages (from other tasks).

void wifi_task(void *arg)
{
    wifi_cmdQueue = xQueueCreate(10,sizeof(wifi_cmdMsg_t));

    wifi_cmdMsg_t msg;

    while(1)
    {
        xQueueReceive(wifi_cmdQueue,&msg,portMAX_DELAY);
        switch(msg.cmd)
        {
            case WIFI_CMD_ENABLE:
                printf("Received wifi enable message\n");
                wifi_enable((cy_wcm_interface_t)msg.arg0);
            break;

        }
    }
}

In the other tasks in the system you “COULD” create a message and submit it to the queue.  I always think that it is easier if you create a function which can be called in the other threads.  Here is the wifi_enable function.  This function takes a char * of either “STA”, “AP”, or “APSTA” and then submits the right message to the queue.

bool wifi_cmd_enable(char *interface)
{
    wifi_cmdMsg_t msg;
    msg.cmd = WIFI_CMD_ENABLE;
    msg.arg0 = (void *)CY_WCM_INTERFACE_TYPE_STA;

    if(strcmp(interface,"STA") == 0)
        msg.arg0 = (void *)CY_WCM_INTERFACE_TYPE_STA;

    else if(strcmp(interface,"AP") == 0)
        msg.arg0 = (void *)CY_WCM_INTERFACE_TYPE_AP;

    else if(strcmp(interface,"APSTA") == 0)
        msg.arg0 = (void *)CY_WCM_INTERFACE_TYPE_AP_STA;
    
    else
    {
        printf("Legal options are STA, AP, APSTA\n");
        return false;
    }

    xQueueSend(wifi_cmdQueue,&msg,0);
    return true;
}

Once I have the nice function for the other tasks, I add it to the public interface in wifi_task.h

#pragma once
#include <stdbool.h>
void wifi_task(void *arg);
bool wifi_cmd_enable(char *interface);

Add a new user command “net”

Now that I have the wifi_task setup I want to add a “net” command to the command line shell.  Start by adding the include.

#include "wifi_task.h"

Then create a function prototype for a new command.

static int usrcmd_net(int argc, char **argv);

Add the command to the list of commands that the shell knows.

static const cmd_table_t cmdlist[] = {
    { "help", "This is a description text string for help command.", usrcmd_help },
    { "info", "This is a description text string for info command.", usrcmd_info },
    { "clear", "Clear the screen", usrcmd_clear },
    { "pargs","print the list of arguments", usrcmd_pargs},
#ifdef configUSE_TRACE_FACILITY 
#if configUSE_STATS_FORMATTING_FUNCTIONS ==1
    { "tasks","print the list of RTOS Tasks", usrcmd_list},
#endif
#endif
    { "net","net [help,enable]",usrcmd_net},
};

Then create the net command.  I want a BUNCH of net commands.  They will include help, enable, connect, disconnect, …. but for now we will start with enable.  This function just calls the wifi_enable command that we added to the wifi_task.h interface.

static int usrcmd_net(int argc, char **argv)
{
    if(argc == 1 || strcmp("help",argv[1]) == 0)
    {
        printf("net [help,enable,connect,disconnect,mdns,scan,ping,lookup]\n");

        printf("%-35s %s\n","net enable","Enable the WiFi Driver & load the WiFi Firmware");

        return 0;
    }

    if(strcmp("enable",argv[1])==0)
    {
            if(argc == 2)
                wifi_cmd_enable("STA");
            else
                wifi_cmd_enable(argv[2]);        
            return 0;
    }

    return 0;
}

Test

Program and test.  Now the “net enable” works.  Notice that it gives you the output about the wifi firmware being loaded into the chip.  Then it tells you that the chip is enabled and the connection manager is rolling.

In the next article I will create a new library of helper functions for wifi.

AnyCloud WiFi Template + WiFi Helper Library (Part 1): Introduction

Summary

The first article in a series that discusses building a new IoT project using Modus Toolbox and the AnyCloud SDK.  Specifically:

  1. The new-ish Error Logging library
  2. AnyCloud Wireless Connection Manager
  3. Creation of New Libraries and Template Projects
  4. Dual Role WiFi Access Point and Station using CYW43012
  5. MDNS

Story

I am working on a new implementation of my Elkhorn Creek IoT monitoring system.  In some of the previous articles I discussed the usage of the Influx Database and Docker as a new cloud backend.  To make this whole thing better I wanted to replace the Raspberry Pi (current system) with a PSoC 6 MCU and a CYW43012 WiFi Chip.  In order to do this, I need to make the PSoC 6 talk to the Influx Database using the WiFi and the Influx DB WebAPI.  I started to build this from my IoT Expert template, but quickly realized that I should make a template project with WiFi.

In this series of article I teach you how to use the Wireless Connection Manager, make new libraries and make new template projects.  Here is the agenda:

Article
(Part 1) Create Basic Project & Add Cypress Logging Functionality
(Part 2) Create New Thread to manage WiFi using the Wireless Connection Manager
(Part 3) Create a New Middleware Library with WiFi helper functions
(Part 4) Add WiFi Scan
Add WiFi Connect
Add WiFi Disconnect
Add WiFi Ping
Add Gethostbyname
Add MDNS
Add Status
Add StartAP
Make a new template project (update manifest)

Create Basic Project

Today I happen to have a CY8CKIT-062S2-43012 on my desk.

So that looks like a good place to start.  Pick that development kit in from the new project creator.

I want to start from my tried and true NT Shell, FreeRTOS Template.  If you use the filter window and type “iot” it will filter things down to just the IoT templates.  Notice that I selected that I want to get a “Microsoft Visual Studio Code” target workspace.

After clicking create you will get a new project.

Something weird happened.  Well actually something bad happened.  When I start Visual Studio Code I get the message that I have multiple workspace files.  Why is that?

So I pick the first one.

Now there is a problem.  In the Makefile for this project I find out that the “APPNAME” is MTBShellTemplate

# Name of application (used to derive name of final linked file).
APPNAME=MTBShellTemplate

By default when you run “make vscode” it will make a workspace file for you with the name “APPNAME.code-workspace”.  This has now created a problem for you.  Specifically, if you regenerate the workspace by running “make vscode” you will update the WRONG file.  When the new project creator runs the “make vscode” it uses the name you entered on that form, not the one in the Makefile.

To fix this, edit he Makefile & delete the old MTB…workspace.  Then re-run make vscode

APPNAME=IoTExpertWiFiTemplate

I have been checking in the *.code-workspace file, but that may not be exactly the right thing to do.  I am not sure.  Oh well.  Here is what you screen should look like now that you have Visual Studio Code going.

I always like to test things to make sure everything works before I start editing.  So, press the play button, then the green play button.

It should build and program the development kit.

Then stop at main.

Press play and your terminal should look something like this.  Notice that I typed “help” and “tasks”

Add the Cypress Logging Functionality

Sometime recently the Software team added a logging capability.  This seems like a good time to try that that.  Start the library browser by running “make modlibs”.  Then enable the “connectivity-utilities”.  For some silly reason that is where the logging functions were added.

If you look in the “mtb_shared” you will now the cy_log directory.

Then click on the “api_reference.html”

And open it.

Cool.  This gives you some insight into the capability.

A simple test will be to printout a “blink” message in sync with the default blinking led.  To do this, I modify the blink_task in main.c  Take the following actions

  1. Add the include “cy_log.h”
  2. Add the initialization call “cy_log_init”
  3. Printout a test message using “cy_log_msg”
  4. Fix the stack
#include "cyhal.h"
#include "cybsp.h"
#include "cy_retarget_io.h"
#include <stdio.h>
#include "FreeRTOS.h"
#include "task.h"
#include "usrcmd.h"
#include "cy_log.h"

volatile int uxTopUsedPriority ;
TaskHandle_t blinkTaskHandle;


void blink_task(void *arg)
{
    cyhal_gpio_init(CYBSP_USER_LED,CYHAL_GPIO_DIR_OUTPUT,CYHAL_GPIO_DRIVE_STRONG,0);

    for(;;)
    {
        cy_log_msg(CYLF_DEF,CY_LOG_INFO,"Blink Info\n");
    	cyhal_gpio_toggle(CYBSP_USER_LED);
    	vTaskDelay(500);
    }
}


int main(void)
{
    uxTopUsedPriority = configMAX_PRIORITIES - 1 ; // enable OpenOCD Thread Debugging

    /* Initialize the device and board peripherals */
    cybsp_init() ;
    __enable_irq();

    cy_retarget_io_init(CYBSP_DEBUG_UART_TX, CYBSP_DEBUG_UART_RX, CY_RETARGET_IO_BAUDRATE);

    cy_log_init(CY_LOG_INFO,0,0);


    // Stack size in WORDs
    // Idle task = priority 0
    xTaskCreate(blink_task, "blinkTask", configMINIMAL_STACK_SIZE*2,0 /* args */ ,0 /* priority */, &blinkTaskHandle);
    xTaskCreate(usrcmd_task, "usrcmd_task", configMINIMAL_STACK_SIZE*4,0 /* args */ ,0 /* priority */, 0);
    vTaskStartScheduler();
}

When you run this, you will get the message repeatedly coming on the screen (probably gonna-want-a delete this before you go on)

Now that we have a working project with logging, in the next article Ill add WiFi

A Standing Desk

Summary

The installation of the AITERMINAL Electric Standing Desk Frame Dual Motor Height Adjustable Desk Motorized Stand Up Desk-White (Frame Only).  Which is only relevant to IoT in that

  1. I worked on this rather than finishing the really cool article that is coming next week.\
  2. It allows me improved place to work

The Story

Last week in California I used a standup desk… which I enjoyed.  I have one at my office in Lexington as well.  But, if you remember from this picture, I don’t have a standing setup at home.

Since my desk is one solid top, I didn’t really know how I could make a change to standup.  But, one afternoon browsing on Amazon I found this:

 

Basically it is a stand-up-desk, but with no table top.  So I ordered it.

 

 

 

 

 

It came in an absolutely giant box.

With a few statistics on the side

The first thing to do is get out the “Modus Toolbox” thanks to my friends in Ukraine

So, my lab assistant got to it.

Then we made a disaster area, by removing all the crap on my desk.

After that we took the tabletop to the barn and ran a track saw on it.

Once the desk was removed… things were REALLY screwed up on the wall.

And the side cabinet.

Here is Nicholas installing the sawed off table top onto the desk.

Here is the desk back in place.  You can see that I have started repairing the drywall.

Which is always an ugly job.

Now that the wall is fixed, Nicholas worked to repair the networking infrastructure.

And here is back together, with the desk in the standing position

And a close up.

It is awesome because it is almost perfectly integrated into the old desk.  Notice that we trimmed about 5 inches off the back of the desk.

The Creek 3.0: Docker & InfluxDB

Summary

Instructions for installing InfluxDB2 in a docker container and writing a Python program to insert data.

Story

I don’t really have a long complicated story about how I got here.  I just wanted to replace my Java, MySQL, Tomcat setup with something newer.  I wanted to do it without writing a bunch of code.  It seemed like Docker + Influx + Telegraph + Grafana was a good answer.  In this article I install Influx DB on my new server using Docker.  Then I hookup my Creek data via a Python script.

Docker & InfluxDB

I have become a huge believer in using Docker, I think it is remarkable what they did.  I also think that using docker-compose is the correct way to launch new docker containers so that you don’t loose the secret sauce on the command line when doing a “docker run”.  Let’s get this whole thing going by creating a new docker-compose.yaml file with the description of our new docker container.  It is pretty simple:

  1. Specify the influxdb image
  2. Map port 8086 on the client and on the container
  3. Specify the initial conditions for the Influxdb – these are nicely documented in the installation instructions here.
  4. Create a volume
version: "3.3"  # optional since v1.27.0
services:
  influxdb:
    image: influxdb
    ports:
      - "8086:8086"
    environment:
      - DOCKER_INFLUXDB_INIT_MODE=setup
      - DOCKER_INFLUXDB_INIT_USERNAME=root
      - DOCKER_INFLUXDB_INIT_PASSWORD=password
      - DOCKER_INFLUXDB_INIT_ORG=creekdata
      - DOCKER_INFLUXDB_INIT_BUCKET=creekdata
    volumes:
      - influxdb2:/var/lib/influxdb2

volumes:
  influxdb2:

Once you have that file you can run “docker-compose up”… and wait … until everything gets pulled from the docker hub.

arh@spiff:~/influx-telegraf-grafana$ docker-compose up
Creating network "influx-telegraf-grafana_default" with the default driver
Creating volume "influx-telegraf-grafana_influxdb2" with default driver
Pulling influxdb (influxdb:)...
latest: Pulling from library/influxdb
d960726af2be: Pull complete
e8d62473a22d: Pull complete
8962bc0fad55: Pull complete
3b26e21cfb07: Pull complete
f77b907603e3: Pull complete
2b137bdfa0c5: Pull complete
7e6fa243fc79: Pull complete
3e0cae572c4f: Pull complete
9a27f9435a76: Pull complete
Digest: sha256:090ba796c2e5c559b9acede14fc7c1394d633fb730046dd2f2ebf400acc22fc0
Status: Downloaded newer image for influxdb:latest
Creating influx-telegraf-grafana_influxdb_1 ... done
Attaching to influx-telegraf-grafana_influxdb_1
influxdb_1  | 2021-05-19T12:37:14.866162317Z	info	booting influxd server in the background	{"system": "docker"}
influxdb_1  | 2021-05-19T12:37:16.867909370Z	info	pinging influxd...	{"system": "docker"}
influxdb_1  | 2021-05-19T12:37:18.879390124Z	info	pinging influxd...	{"system": "docker"}
influxdb_1  | 2021-05-19T12:37:20.891280023Z	info	pinging influxd...	{"system": "docker"}
influxdb_1  | ts=2021-05-19T12:37:21.065674Z lvl=info msg="Welcome to InfluxDB" log_id=0UD9wCAG000 version=2.0.6 commit=4db98b4c9a build_date=2021-04-29T16:48:12Z
influxdb_1  | ts=2021-05-19T12:37:21.068517Z lvl=info msg="Resources opened" log_id=0UD9wCAG000 service=bolt path=/var/lib/influxdb2/influxd.bolt
influxdb_1  | ts=2021-05-19T12:37:21.069293Z lvl=info msg="Bringing up metadata migrations" log_id=0UD9wCAG000 service=migrations migration_count=15
influxdb_1  | ts=2021-05-19T12:37:21.132269Z lvl=info msg="Using data dir" log_id=0UD9wCAG000 service=storage-engine service=store path=/var/lib/influxdb2/engine/data
influxdb_1  | ts=2021-05-19T12:37:21.132313Z lvl=info msg="Compaction settings" log_id=0UD9wCAG000 service=storage-engine service=store max_concurrent_compactions=3 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
influxdb_1  | ts=2021-05-19T12:37:21.132325Z lvl=info msg="Open store (start)" log_id=0UD9wCAG000 service=storage-engine service=store op_name=tsdb_open op_event=start
influxdb_1  | ts=2021-05-19T12:37:21.132383Z lvl=info msg="Open store (end)" log_id=0UD9wCAG000 service=storage-engine service=store op_name=tsdb_open op_event=end op_elapsed=0.059ms
influxdb_1  | ts=2021-05-19T12:37:21.132407Z lvl=info msg="Starting retention policy enforcement service" log_id=0UD9wCAG000 service=retention check_interval=30m
influxdb_1  | ts=2021-05-19T12:37:21.132428Z lvl=info msg="Starting precreation service" log_id=0UD9wCAG000 service=shard-precreation check_interval=10m advance_period=30m
influxdb_1  | ts=2021-05-19T12:37:21.132446Z lvl=info msg="Starting query controller" log_id=0UD9wCAG000 service=storage-reads concurrency_quota=1024 initial_memory_bytes_quota_per_query=9223372036854775807 memory_bytes_quota_per_query=9223372036854775807 max_memory_bytes=0 queue_size=1024
influxdb_1  | ts=2021-05-19T12:37:21.133391Z lvl=info msg="Configuring InfluxQL statement executor (zeros indicate unlimited)." log_id=0UD9wCAG000 max_select_point=0 max_select_series=0 max_select_buckets=0
influxdb_1  | ts=2021-05-19T12:37:21.434078Z lvl=info msg=Starting log_id=0UD9wCAG000 service=telemetry interval=8h
influxdb_1  | ts=2021-05-19T12:37:21.434165Z lvl=info msg=Listening log_id=0UD9wCAG000 service=tcp-listener transport=http addr=:9999 port=9999
influxdb_1  | 2021-05-19T12:37:22.905008706Z	info	pinging influxd...	{"system": "docker"}
influxdb_1  | 2021-05-19T12:37:22.920976742Z	info	got response from influxd, proceeding	{"system": "docker"}
influxdb_1  | Config default has been stored in /etc/influxdb2/influx-configs.
influxdb_1  | User	Organization	Bucket
influxdb_1  | root	creekdata	creekdata
influxdb_1  | 2021-05-19T12:37:23.043336133Z	info	Executing user-provided scripts	{"system": "docker", "script_dir": "/docker-entrypoint-initdb.d"}
influxdb_1  | 2021-05-19T12:37:23.044663106Z	info	initialization complete, shutting down background influxd	{"system": "docker"}
influxdb_1  | ts=2021-05-19T12:37:23.044900Z lvl=info msg="Terminating precreation service" log_id=0UD9wCAG000 service=shard-precreation
influxdb_1  | ts=2021-05-19T12:37:23.044906Z lvl=info msg=Stopping log_id=0UD9wCAG000 service=telemetry interval=8h
influxdb_1  | ts=2021-05-19T12:37:23.044920Z lvl=info msg=Stopping log_id=0UD9wCAG000 service=scraper
influxdb_1  | ts=2021-05-19T12:37:23.044970Z lvl=info msg=Stopping log_id=0UD9wCAG000 service=tcp-listener
influxdb_1  | ts=2021-05-19T12:37:23.545252Z lvl=info msg=Stopping log_id=0UD9wCAG000 service=task
influxdb_1  | ts=2021-05-19T12:37:23.545875Z lvl=info msg=Stopping log_id=0UD9wCAG000 service=nats
influxdb_1  | ts=2021-05-19T12:37:23.546765Z lvl=info msg=Stopping log_id=0UD9wCAG000 service=bolt
influxdb_1  | ts=2021-05-19T12:37:23.546883Z lvl=info msg=Stopping log_id=0UD9wCAG000 service=query
influxdb_1  | ts=2021-05-19T12:37:23.548747Z lvl=info msg=Stopping log_id=0UD9wCAG000 service=storage-engine
influxdb_1  | ts=2021-05-19T12:37:23.548788Z lvl=info msg="Closing retention policy enforcement service" log_id=0UD9wCAG000 service=retention
influxdb_1  | ts=2021-05-19T12:37:29.740107Z lvl=info msg="Welcome to InfluxDB" log_id=0UD9wj2l000 version=2.0.6 commit=4db98b4c9a build_date=2021-04-29T16:48:12Z
influxdb_1  | ts=2021-05-19T12:37:29.751816Z lvl=info msg="Resources opened" log_id=0UD9wj2l000 service=bolt path=/var/lib/influxdb2/influxd.bolt
influxdb_1  | ts=2021-05-19T12:37:29.756974Z lvl=info msg="Checking InfluxDB metadata for prior version." log_id=0UD9wj2l000 bolt_path=/var/lib/influxdb2/influxd.bolt
influxdb_1  | ts=2021-05-19T12:37:29.757053Z lvl=info msg="Using data dir" log_id=0UD9wj2l000 service=storage-engine service=store path=/var/lib/influxdb2/engine/data
influxdb_1  | ts=2021-05-19T12:37:29.757087Z lvl=info msg="Compaction settings" log_id=0UD9wj2l000 service=storage-engine service=store max_concurrent_compactions=3 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
influxdb_1  | ts=2021-05-19T12:37:29.757099Z lvl=info msg="Open store (start)" log_id=0UD9wj2l000 service=storage-engine service=store op_name=tsdb_open op_event=start
influxdb_1  | ts=2021-05-19T12:37:29.757149Z lvl=info msg="Open store (end)" log_id=0UD9wj2l000 service=storage-engine service=store op_name=tsdb_open op_event=end op_elapsed=0.051ms
influxdb_1  | ts=2021-05-19T12:37:29.757182Z lvl=info msg="Starting retention policy enforcement service" log_id=0UD9wj2l000 service=retention check_interval=30m
influxdb_1  | ts=2021-05-19T12:37:29.757187Z lvl=info msg="Starting precreation service" log_id=0UD9wj2l000 service=shard-precreation check_interval=10m advance_period=30m
influxdb_1  | ts=2021-05-19T12:37:29.757205Z lvl=info msg="Starting query controller" log_id=0UD9wj2l000 service=storage-reads concurrency_quota=1024 initial_memory_bytes_quota_per_query=9223372036854775807 memory_bytes_quota_per_query=9223372036854775807 max_memory_bytes=0 queue_size=1024
influxdb_1  | ts=2021-05-19T12:37:29.758844Z lvl=info msg="Configuring InfluxQL statement executor (zeros indicate unlimited)." log_id=0UD9wj2l000 max_select_point=0 max_select_series=0 max_select_buckets=0
influxdb_1  | ts=2021-05-19T12:37:30.056855Z lvl=info msg=Listening log_id=0UD9wj2l000 service=tcp-listener transport=http addr=:8086 port=8086
influxdb_1  | ts=2021-05-19T12:37:30.056882Z lvl=info msg=Starting log_id=0UD9wj2l000 service=telemetry interval=8h

After everything is rolling you can open up a web browser and go to “http://localhost:8086” and you should see something like this:  (I will sort out the http vs https in a later post – because I don’t actually know how to fix it right now.

Once you enter the account and password (that you configured in the docker-compose.yaml” you will see this screen and you are off to the races.

InfluxDB Basics

Before we go too much further lets talk about some of the basics of the Influx Database.  An Influx Database also called a “bucket” has the following built in columns:

  • _timestamp: The time for the data point stored in epoch nanosecond format (how’s that for some precision)
  • _measurement: A text string name for the a group of related datapoints
  • _field: A text string key for the datapoint
  • _value: The value of the datapoint

In addition you can add “ad-hoc” columns called “tags” which have a “key” and a “value”

Organization A group of users and the related buckets, dashboards and tasks
Bucket A database
Timestamp The time of the datapoint measured in epoch nanoseconds
Field A field includes a field key stored in the _field column and a field value stored in the _value column.
Field Set A field set is a collection of field key-value pairs associated with a timestamp.
Measurement A measurement acts as a container for tags fields and timestamps. Use a measurement name that describes your data.
Tag Key/Value pairs assigned to a datapoint.  They are used to index the datapoints (so searches are faster)

Here is a snapshot of the data in my Creek Influx database.  You can see that I have two fields

  • depth
  • temperature

I am saving all of the datapoints in the “elkhorncreek” _measurement.  And there are no tags (but I have ideas for that in the future)

InfluxDB Line Protocol

There are a number of different methods to insert data into the Influx DB.  Several of them rely on “Line Protocol“.  This is simply a text string formatted like this:

For my purposes a text string like this will insert a new datapoint into the “elkhorncreek” measurement with a depth of 1.85 fee and a temperature of 19c (yes we are a mixed unit household)

  • elkhorncreek depth=1.85,temperature=19.0

Python & InfluxDB

I know that I want to run a Python program on the Raspberry Pi which gets the sensor data via I2C and then writes it into the cloud using the InfluxAPI.  It turns out that when you log into you new Influx DB that there is a built in webpage which shows you exactly how to do this.  Click on “Data” then “sources” then “Python”

You will see a screen like this which has exactly the Python code you need (almost).

To make this code work on your system you need to install the influxdb-client library by running “pip install influxdb-client”

(venv) pi@iotexpertpi:~/influx-test $ pip install influxdb-client
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting influxdb-client
  Using cached https://files.pythonhosted.org/packages/6b/0e/5c5a9a2da144fae80b23dd9741175493d8dbeabd17d23e5aff27c92dbfd5/influxdb_client-1.17.0-py3-none-any.whl
Collecting urllib3>=1.15.1 (from influxdb-client)
  Using cached https://files.pythonhosted.org/packages/09/c6/d3e3abe5b4f4f16cf0dfc9240ab7ce10c2baa0e268989a4e3ec19e90c84e/urllib3-1.26.4-py2.py3-none-any.whl
Collecting pytz>=2019.1 (from influxdb-client)
  Using cached https://files.pythonhosted.org/packages/70/94/784178ca5dd892a98f113cdd923372024dc04b8d40abe77ca76b5fb90ca6/pytz-2021.1-py2.py3-none-any.whl
Collecting certifi>=14.05.14 (from influxdb-client)
  Using cached https://files.pythonhosted.org/packages/5e/a0/5f06e1e1d463903cf0c0eebeb751791119ed7a4b3737fdc9a77f1cdfb51f/certifi-2020.12.5-py2.py3-none-any.whl
Collecting rx>=3.0.1 (from influxdb-client)
  Using cached https://files.pythonhosted.org/packages/e2/a9/efeaeca4928a9a56d04d609b5730994d610c82cf4d9dd7aa173e6ef4233e/Rx-3.2.0-py3-none-any.whl
Collecting six>=1.10 (from influxdb-client)
  Using cached https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=21.0.0 in ./venv/lib/python3.7/site-packages (from influxdb-client) (40.8.0)
Collecting python-dateutil>=2.5.3 (from influxdb-client)
  Using cached https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl
Installing collected packages: urllib3, pytz, certifi, rx, six, python-dateutil, influxdb-client
Successfully installed certifi-2020.12.5 influxdb-client-1.17.0 python-dateutil-2.8.1 pytz-2021.1 rx-3.2.0 six-1.16.0 urllib3-1.26.4
(venv) pi@iotexpertpi:~/influx-test $

Now write a little bit of code.  If you remember from the previous post I run a cronjob that gets the data from the I2C.  It will then run this program to do the insert of the data into the Influxdb.  Notice that I get the depth and temperature from the command line.   The “token” is an API key which you must include with requests to identify you are having permission to write into the database (more on this later).  The “data” variable is just a string formatted in “Influx Line Protocol”

import sys
from datetime import datetime
from influxdb_client import InfluxDBClient, Point, WritePrecision
from influxdb_client.client.write_api import SYNCHRONOUS

if len(sys.argv) != 3:
    sys.exit("Wrong number of arguments")

# You can generate a Token from the "Tokens Tab" in the UI
token = "UvZvrrnk8yXvlVm1yrMmH2ZE706dZ14kpqSoE2u0COnDdqmQFTmIWPMjk0U2tO_GqmjzCupi_EaYP65RP4bELQ=="
org = "creekdata"
bucket = "creekdata"

client = InfluxDBClient(url="http://linux.local:8086", token=token)

write_api = client.write_api(write_options=SYNCHRONOUS)

data = f"elkhorncreek depth={sys.argv[1]},temperature={sys.argv[2]}"
write_api.write(bucket, org, data)
#print(data)

Now I update my getInsertData.sh shell script to run the Influx as well as the original MySQL insert.

#!/bin/bash

cd ~/influxdb
source venv/bin/activate
vals=$(python getData.py)
#echo $vals
python insertMysql.py $vals
python insertInflux.py $vals

InfluxDB Data Explorer

After a bit of time (for some inserts to happen) I go to the data explorer in the web interface.  You can see that I have a number of readings.  This is filtering for “depth”

This is filtering for “temperature”

Influx Tokens

To interact with an instance of the InfluxDB you will need an API key, which they call a token.  Press the “data” icon on the left side of the screen.  Then click “Tokens”.  You will see the currently available tokens, in this case just the original token.  You can create more tokens by pressing the blue + generate Token icon.

Clock on the token.  Then copy it to your clipboard.

Influx DB CLI Making Me Crazy

Summary

A solution to the Influx DB CLI error “Failed to check token: received status code 401 from server” including instructions to install the InfluxDB V2 CLI.

Story

As I worked my way through using the Influx Database I tried using the InfluxDB CLI.  However, no matter what I seemed to do I got this error, which was super annoying.

linux$ 
linux$ influx
Failed to check token: received status code 401 from server
linux$ influx create
Failed to check token: received status code 401 from server
linux$ influx adsf
Failed to check token: received status code 401 from server
linux$ influx setup
Failed to check token: received status code 401 from server

I tried googling around to try to figure out what was happening but really didn’t see anything that would explain that behavior.  However, I did find one comment that if you were using Docker you could run the CLI by running in the shell.  So I tried that:

linux$ docker exec -it 89ffd4bb9ec5  /bin/bash
root@89ffd4bb9ec5:/# influx version
Influx CLI 2.0.4 (git: 4e7a59bb9a) build_date: 2021-02-08T17:47:02Z

When I originally installed the client (on Ubuntu) I did this:

linux$ sudo apt install influxdb-client
[sudo] password for arh: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  linux-hwe-5.4-headers-5.4.0-71
Use 'sudo apt autoremove' to remove it.
The following NEW packages will be installed:
  influxdb-client
0 upgraded, 1 newly installed, 0 to remove and 97 not upgraded.
Need to get 1,146 kB of archives.
After this operation, 3,969 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 influxdb-client amd64 1.1.1+dfsg1-4 [1,146 kB]
Fetched 1,146 kB in 1s (1,883 kB/s)       
Selecting previously unselected package influxdb-client.
(Reading database ... 275845 files and directories currently installed.)
Preparing to unpack .../influxdb-client_1.1.1+dfsg1-4_amd64.deb ...
Unpacking influxdb-client (1.1.1+dfsg1-4) ...
Setting up influxdb-client (1.1.1+dfsg1-4) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

When I ran the version look what I got:

linux$ /usr/bin/influx -version
InfluxDB shell version: 1.1.1

Now we know the problem.  I have the wrong version of the CLI.  This was caused by two things

  1. I installed the docker version of InfluxDB.  So I never had the command line version on my linux box.
  2. The influxdb-client that you get from whatever the debian source on Ubuntu is the 1.xx version.  It is too bad that you can’t “sudo apt install influxdbv2-client”  Oh well.

To fix this start by nuking the old version.

linux$ sudo apt-get remove influxdb-client
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  linux-hwe-5.4-headers-5.4.0-71
Use 'sudo apt autoremove' to remove it.
The following packages will be REMOVED:
  influxdb-client
0 upgraded, 0 newly installed, 1 to remove and 97 not upgraded.
After this operation, 3,969 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 275851 files and directories currently installed.)
Removing influxdb-client (1.1.1+dfsg1-4) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

Then download the “InfluxDB Cloud CLI” (which is just a standalone version of the CLI.  You can get it from the influxdata.com website here.

Scroll down to the InfluxDB Cloud CLI

Then follow the instructions (Notice that I moved it to /usr/local/bin)

linux$ wget https://dl.influxdata.com/influxdb/releases/influxdb2-client-2.0.6-linux-amd64.tar.gz
--2021-05-21 11:08:48--  https://dl.influxdata.com/influxdb/releases/influxdb2-client-2.0.6-linux-amd64.tar.gz
Resolving dl.influxdata.com (dl.influxdata.com)... 13.33.74.100, 13.33.74.21, 13.33.74.27, ...
Connecting to dl.influxdata.com (dl.influxdata.com)|13.33.74.100|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11758006 (11M) [application/x-gzip]
Saving to: ‘influxdb2-client-2.0.6-linux-amd64.tar.gz’

influxdb2-client-2.0.6-linux-amd64.tar.gz              100%[===========================================================================================================================>]  11.21M  10.8MB/s    in 1.0s    

2021-05-21 11:08:50 (10.8 MB/s) - ‘influxdb2-client-2.0.6-linux-amd64.tar.gz’ saved [11758006/11758006]

linux$ tar xvf influxdb2-client-2.0.6-linux-amd64.tar.gz 
influxdb2-client-2.0.6-linux-amd64/LICENSE
influxdb2-client-2.0.6-linux-amd64/README.md
influxdb2-client-2.0.6-linux-amd64/influx
linux$ sudo mv influxdb2-client-2.0.6-linux-amd64/influx /usr/local/bin

Now when you check the version, you are in the money

linux$ influx version
Influx CLI 2.0.6 (git: 4db98b4c9a) build_date: 2021-04-29T16:48:12Z

The Creek 3.0: A Docker MySQL Diversion – Part 2.5

Summary

A discussion of reading I2C data from a sensor and sending it to a MySQL instance in the cloud using Python.

I was originally planning only one article on the MySQL part of this project.  But things got really out of control and I ended up splitting the article into two parts.  I jokingly called this article “Part 2.5”.  In today’s article I’ll take the steps to have Python and the libraries running on the Raspberry Pi to read data and send it to my new Docker MySQL Server.

Here is what the picture looks like:

Build the Python Environment w/smbus & mysql-connector-python

I typically like to build a Python virtual environment with the specific version of python and all of the required packages.  To do this you need to

  1. python3 -m venv venv
  2. source venv/bin/activate
  3. pip install smbus
  4. pip install mysql-connector-python
pi@iotexpertpi:~ $ mkdir mysql-docker
pi@iotexpertpi:~ $ python3 -m venv venv
pi@iotexpertpi:~ $ source venv/bin/activate
(venv) pi@iotexpertpi:~ $ pip install smbus
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting smbus
  Using cached https://www.piwheels.org/simple/smbus/smbus-1.1.post2-cp37-cp37m-linux_armv6l.whl
Installing collected packages: smbus
Successfully installed smbus-1.1.post2
(venv) pi@iotexpertpi:~ $ pip install mysql-connector-python
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting mysql-connector-python
  Using cached https://files.pythonhosted.org/packages/2a/8a/428d6be58fab7106ab1cacfde3076162cd3621ef7fc6871da54da15d857d/mysql_connector_python-8.0.25-py2.py3-none-any.whl
Collecting protobuf>=3.0.0 (from mysql-connector-python)
  Downloading https://files.pythonhosted.org/packages/6b/2c/62cee2a27a1c4c0189582330774ed6ac2bfc88cb223f04723620ee04d59d/protobuf-3.17.0-py2.py3-none-any.whl (173kB)
    100% |████████████████████████████████| 174kB 232kB/s 
Collecting six>=1.9 (from protobuf>=3.0.0->mysql-connector-python)
  Using cached https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl
Installing collected packages: six, protobuf, mysql-connector-python
Successfully installed mysql-connector-python-8.0.25 protobuf-3.17.0 six-1.16.0
(venv) pi@iotexpertpi:~

Once that is done you can see that everything is copasetic by running “pip freeze” where you can see the mysql-connector-python and the smbus.

(venv) pi@iotexpertpi:~ $ pip freeze
mysql-connector-python==8.0.25
pkg-resources==0.0.0
protobuf==3.17.0
six==1.16.0
smbus==1.1.post2

Python: Get Data SMBUS

If you remember from the original design that the PSoC 4 acts as a register file with the data from the temperature and pressure sensor.  It has 12 bytes of data as

  1. 2-bytes formatted as a 16-bit unsigned ADC counts from the Pressure Sensor
  2. 2-bytes formatted as a 16-bit signed pressure in “centiTemp”
  3. 4-bytes float as the depth in Feet
  4. 4-bytes float as the temperature in Centigrade

This program:

  1. Reads the I2c for 12-bytes
  2. Converts it into an array
  3. Prints out the values
import struct
import sys
import smbus
from datetime import datetime
from influxdb_client import InfluxDBClient, Point, WritePrecision
from influxdb_client.client.write_api import SYNCHRONOUS


######################################################
#Read the data from the PSoC 4
######################################################
bus = smbus.SMBus(1)
address = 0x08

# The data structure in the PSOC 4 is:
# uint16_t pressureCount ; the adc-counts being read on the pressure sensor
# int16_t centiTemp ; the temperaure in 10ths of a degree C
# float depth ; four bytes float representing the depth in Feet
# float temperature ; four byte float representing the temperature in degrees C

numBytesInStruct = 12
block = bus.read_i2c_block_data(address, 0, numBytesInStruct)

# convert list of bytes returned from sensor into array of bytes
mybytes = bytearray(block)
# convert the byte array into
# H=Unsigned 16-bit int
# h=Signed 16-bit int
# f=Float 
# this function will return a tuple with pressureCount,centiTemp,depth,temperature
vals = struct.unpack_from('Hhff',mybytes,0)
# prints the tuple
depth = vals[2]
temperature = vals[3]
print(f"{depth} {temperature}")

Python: MySQL

I created a separate Python program to insert the data into the MySQL database.  This program does the following things

  1. Makes sure the command line arguments make sense
  2. Makes a connection to the server
  3. Creates the SQL statement
  4. Runs the inserts
import mysql.connector
import sys
from datetime import datetime


if len(sys.argv) != 3:
    sys.exit("Wrong number of arguments")

mydb = mysql.connector.connect(
    host="spiff.local",
    user="creek",
    password="donthackme",
    database="creekdata",
    auth_plugin='mysql_native_password')

now = datetime.now()
formatted_date = now.strftime('%Y-%m-%d %H:%M:%S')
sql = "insert into creekdata.creekdata (depth,temperature,created_at) values (%s,%s,%s)"
vals = (sys.argv[1],sys.argv[2],formatted_date)


mycursor = mydb.cursor()

mycursor.execute(sql, vals)

mydb.commit()

Shell Script & Crontab

I created a simple bash shell script to

  1. Activate the virtual enviroment
  2. Run the get data python program
  3. Run the insert program
#!/bin/bash

cd ~/influxdb
source venv/bin/activate
vals=$(python getData.py)
#echo $vals
python insertMysql.py $vals

Finally, a cronjob to run the program every 5 minutes.

# Edit this file to introduce tasks to be run by cron.
# 
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command
0,5,10,20,25,30,35,40,45,50,55 * * * * /home/pi/influxdb/getInsertData.sh

Test with MySQL WorkBench

Now when I load the data from the MySQL Workbench I can see the inserts are happening.  Kick ass.

The Creek 3.0: A Docker MySQL Diversion – Part 2

Summary

A tutorial on running MySQL in an instance of Docker on Ubuntu Linux.  Then creating a Raspberry Pi Python interface from a sensor to insert data over the network to the new MySQL Server.

Story

As I said in the introduction, this whole process has been a bit chaotic.  So here we go.  The Raspberry Pi that runs the current creek system has been in my barn since at least 2013 running on the same SD Card and never backed up.  I suppose that it wouldn’t have really mattered if I lost the old flood data, but it would have been annoying.  Also, that Raspberry Pi is very slow running queries given the 2.2M records that now exist in the database.

To fix this I decided that I want to start by moving the MySQL server to a new computer that runs Docker.  Here is the original configuration (from the original article)

When I set out to do in this article the plan was to move the MySQL Instance from the Raspberry Pi to a new Linux box.  Unfortunately while I was doing this, I broke the operating system on the Raspberry Pi and ended up having to rebuild the interface to the PSoC 4.  Here is what I ended up building:

This article will walk you through the following steps.

  1. Build a new Linux machine & Install Ubuntu Server
  2. Install Docker & MySQL
  3. Migrate the Data from the original Raspberry Pi MySQL Database
  4. Build the Python Environment (Part 2.5)
  5. Python: Get Data SMBUS (Part 2.5)
  6. Python: Insert MySQL (Part 2.5)
  7. Shell Script & Crontab (Part 2.5)
  8. Test using MySQL WorkBench (Part 2.5)

Build a new Linux Box with Ubuntu Server

I wanted to have a local to my lan server running MySQL.  My lab assistant suggested that I find something fairly inexpensive on ebay.  Here is what I bought:

 

And… for sure it needed an SSD.

Then I downloaded Ubuntu Server 20.04 from https://ubuntu.com/download/server

After the file was downloaded I created a bootable sdcard by running: dd if=ubuntu-20.04.2-live-server-amd64.iso of=/dev/rdisk4 bs=1m

arh Downloads $ sudo diskutil unmountDisk /dev/disk4
Unmount of all volumes on disk4 was successful
arh Downloads $ sudo dd if=ubuntu-20.04.2-live-server-amd64.iso of=/dev/rdisk4 bs=1m
1158+1 records in
1158+1 records out
1215168512 bytes transferred in 32.400378 secs (37504763 bytes/sec)
arh Downloads $ diskutil list /dev/disk4
/dev/disk4 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     Apple_partition_scheme                        *31.1 GB    disk4
   1:        Apple_partition_map ⁨⁩                        4.1 KB     disk4s1
   2:                  Apple_HFS ⁨⁩                        4.1 MB     disk4s2

After doing the installation (I dont have screen captures of that, but it is easy).  I installed the avahi daemon.  What is that?  Avahi is program that enables mDNS – a part of no configuration networking that helps you manage “names”.  Specifically in my case it will create a DNS-like name for this computer without having to actually configure the DNS.  That name is “linux.local”.

To install avahi run sudo apt install avahi-daemon

arh@spiff:~$ systemctl status avahi-daemon
● avahi-daemon.service - Avahi mDNS/DNS-SD Stack
     Loaded: loaded (/lib/systemd/system/avahi-daemon.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-05-01 14:20:40 UTC; 2 weeks 1 days ago
TriggeredBy: ● avahi-daemon.socket
   Main PID: 713 (avahi-daemon)
     Status: "avahi-daemon 0.7 starting up."
      Tasks: 2 (limit: 14105)
     Memory: 2.8M
     CGroup: /system.slice/avahi-daemon.service
             ├─713 avahi-daemon: running [spiff.local]
             └─757 avahi-daemon: chroot helper

May 11 11:26:26 spiff avahi-daemon[713]: Registering new address record for fe80::4409:73ff:fe08:4c75 on veth72ac3b7.*.
May 11 11:26:26 spiff avahi-daemon[713]: Joining mDNS multicast group on interface br-18a7431f8090.IPv6 with address fe80::42:beff:fe8c:e24.
May 11 11:26:26 spiff avahi-daemon[713]: New relevant interface br-18a7431f8090.IPv6 for mDNS.
May 11 11:26:26 spiff avahi-daemon[713]: Registering new address record for fe80::42:beff:fe8c:e24 on br-18a7431f8090.*.
May 11 11:26:43 spiff avahi-daemon[713]: Interface veth72ac3b7.IPv6 no longer relevant for mDNS.
May 11 11:26:43 spiff avahi-daemon[713]: Leaving mDNS multicast group on interface veth72ac3b7.IPv6 with address fe80::4409:73ff:fe08:4c75.
May 11 11:26:43 spiff avahi-daemon[713]: Withdrawing address record for fe80::4409:73ff:fe08:4c75 on veth72ac3b7.
May 11 11:26:48 spiff avahi-daemon[713]: Joining mDNS multicast group on interface veth5c71e0d.IPv6 with address fe80::4499:b0ff:feef:30fe.
May 11 11:26:48 spiff avahi-daemon[713]: New relevant interface veth5c71e0d.IPv6 for mDNS.
May 11 11:26:48 spiff avahi-daemon[713]: Registering new address record for fe80::4499:b0ff:feef:30fe on veth5c71e0d.*.
arh@spiff:~$ 

I also will be running MySQL in a Docker instance.  To install docker run: sudo apt install docker.io

arh@spiff:~$ systemctl status docker
● docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2021-05-01 14:20:42 UTC; 2 weeks 1 days ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
   Main PID: 758 (dockerd)
      Tasks: 26
     Memory: 142.1M
     CGroup: /system.slice/docker.service
             ├─   758 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
             └─240639 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 3306 -container-ip 172.18.0.2 -container-port 3306

May 02 13:39:42 spiff dockerd[758]: time="2021-05-02T13:39:42.754047181Z" level=info msg="ignoring event" container=4d2e6a3c8c779e01676e4fd8f748aa4581c9469d92398ff274a3800c5d3e98a2 module>
May 02 13:40:42 spiff dockerd[758]: time="2021-05-02T13:40:42.381681852Z" level=error msg="Error setting up exec command in container 4d2e6a3c8c77: Container 4d2e6a3c8c779e01676e4fd8f748a>
May 02 13:40:42 spiff dockerd[758]: time="2021-05-02T13:40:42.760184585Z" level=warning msg="error locating sandbox id 5e4b44ba78eacdb974bfd773ffabf46526177f4ff135ace09b667c3e497b3468: sa>
May 02 13:40:42 spiff dockerd[758]: time="2021-05-02T13:40:42.762228692Z" level=error msg="4d2e6a3c8c779e01676e4fd8f748aa4581c9469d92398ff274a3800c5d3e98a2 cleanup: failed to delete conta>
May 02 13:40:42 spiff dockerd[758]: time="2021-05-02T13:40:42.764274310Z" level=error msg="restartmanger wait error: network c6593d532df7651e3a38572e609d42f69f0daba3ac36263933ca0ae43504cc>
May 11 11:26:24 spiff dockerd[758]: time="2021-05-11T11:26:24.772660650Z" level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [namese>
May 11 11:26:24 spiff dockerd[758]: time="2021-05-11T11:26:24.772680359Z" level=info msg="IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2>
May 11 11:26:42 spiff dockerd[758]: time="2021-05-11T11:26:42.994091472Z" level=info msg="ignoring event" container=bfd550cab791b061bbd4e26f3435165de7b3664373de9cbb80d2e78a0aff08e2 module>
May 11 11:26:46 spiff dockerd[758]: time="2021-05-11T11:26:46.212688536Z" level=info msg="No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [namese>
May 11 11:26:46 spiff dockerd[758]: time="2021-05-11T11:26:46.212708396Z" level=info msg="IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2>
arh@spiff:~$

Docker Training

I knew that I wanted to try Docker, no kidding eh, but I didn’t know much of anything about it.  I am not really a “video” person for learning, but my son had talked me into trying a skill share class to learn how to edit video.  So, I thought that I would give it a try for Docker as well.  This class was OK but not great (like 2/5)  Here is a screenshot from the class:

I also watched this class, which is excellent…. especially if you watch it at 1.5x speed.

Docker Introduction

There are four basic ideas which you need to understand Docker.

Concept Description Commands
Image A runnable binary template that can be instantiated into a container (like a class in object oriented programming) docker image ls
Container An VM-like instance of an image (like an object i.e. an instance of a class in object oriented programming).  This includes network port mapping, volumes,network etc. docker ps -a
Volume A directory or file map between the host operating system and the docker container.  For example a directory X on the host is mapped to the directory Y inside of the container docker volume ls
Network A synthetic network that is created by the docker daemon to map one or more containers together.  This includes a dhcp, dns, routing etc. docker network ls

Docker Compose & MySQL

You can find new images at https://hub.docker.com.  In fact this is where I get everything that I need for mysql.

If you look a little bit later down on the docker hub you will find the specific instruction for “running” a docker mysql image.

These instructions will work.  However, there are two problems.

#1 by running it this way you will not expose the ip port 3306 from inside of the container to the outside work (on your computer or network).  This means you won’t be able to talk to the MySQL instance.  That is not very helpful

#2 all of the secret sauce you typed will be lost if you need to do that same command again.

The good news is that docker has a specific file format for saving this information called “docker-compose.yaml”.

My docker compose file looks like this.

  1. The image is “mysql” (use the official docker mysql image)
  2. Map the MySQL port 3306 from inside the container to the outside
  3. Make the root password “supersecret”
  4. Create a database called “creekdata”
  5. Create a user called “creek” with a password “asillypassword”
  6. Map the mysql data inside of the container at /var/lib/mysql to an outside volume called “mysql”
arh@spiff:~/mysql$ more docker-compose.yaml
version: '3.1'

services:

  db:
    image: mysql
    restart: always
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: supersecret
      MYSQL_DATABASE: creekdata
      MYSQL_USER: creek
      MYSQL_PASSWORD: asillypassword
    volumes:
      - mysql:/var/lib/mysql
      
volumes:
  mysql:

With this file I can create the container by running “docker-compose up”

linux$ docker-compose up
Creating network "mysql_default" with the default driver
Creating volume "mysql_mysql" with default driver
Pulling db (mysql:latest)...
latest: Pulling from library/mysql
69692152171a: Pull complete
1651b0be3df3: Pull complete
951da7386bc8: Pull complete
0f86c95aa242: Pull complete
37ba2d8bd4fe: Pull complete
6d278bb05e94: Pull complete
497efbd93a3e: Pull complete
f7fddf10c2c2: Pull complete
16415d159dfb: Pull complete
0e530ffc6b73: Pull complete
b0a4a1a77178: Pull complete
cd90f92aa9ef: Pull complete
Digest: sha256:d50098d7fcb25b1fcb24e2d3247cae3fc55815d64fec640dc395840f8fa80969
Status: Downloaded newer image for mysql:latest
Creating mysql_db_1 ... 
Creating mysql_db_1 ... done
Attaching to mysql_db_1
db_1  | 2021-05-17 20:01:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.25-1debian10 started.
db_1  | 2021-05-17 20:01:20+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
db_1  | 2021-05-17 20:01:20+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.25-1debian10 started.
db_1  | 2021-05-17 20:01:20+00:00 [Note] [Entrypoint]: Initializing database files
db_1  | 2021-05-17T20:01:20.192621Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.25) initializing of server in progress as process 41
db_1  | 2021-05-17T20:01:20.196027Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
db_1  | 2021-05-17T20:01:20.770999Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
db_1  | 2021-05-17T20:01:21.809117Z 6 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
db_1  | 2021-05-17 20:01:24+00:00 [Note] [Entrypoint]: Database files initialized
db_1  | 2021-05-17 20:01:24+00:00 [Note] [Entrypoint]: Starting temporary server
db_1  | 2021-05-17T20:01:24.396505Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.25) starting as process 86
db_1  | 2021-05-17T20:01:24.415784Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
db_1  | 2021-05-17T20:01:24.551463Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
db_1  | 2021-05-17T20:01:24.618191Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
db_1  | 2021-05-17T20:01:24.726805Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
db_1  | 2021-05-17T20:01:24.726923Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
db_1  | 2021-05-17T20:01:24.728714Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
db_1  | 2021-05-17T20:01:24.738807Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.25'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server - GPL.
db_1  | 2021-05-17 20:01:24+00:00 [Note] [Entrypoint]: Temporary server started.
db_1  | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
db_1  | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
db_1  | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
db_1  | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
db_1  | 2021-05-17 20:01:25+00:00 [Note] [Entrypoint]: Creating database creekdata
db_1  | 2021-05-17 20:01:25+00:00 [Note] [Entrypoint]: Creating user creek
db_1  | 2021-05-17 20:01:25+00:00 [Note] [Entrypoint]: Giving user creek access to schema creekdata
db_1  | 
db_1  | 2021-05-17 20:01:25+00:00 [Note] [Entrypoint]: Stopping temporary server
db_1  | 2021-05-17T20:01:25.775184Z 13 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.25).
db_1  | 2021-05-17T20:01:27.490685Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.25)  MySQL Community Server - GPL.
db_1  | 2021-05-17 20:01:27+00:00 [Note] [Entrypoint]: Temporary server stopped
db_1  | 
db_1  | 2021-05-17 20:01:27+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.
db_1  | 
db_1  | 2021-05-17T20:01:27.988961Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.25) starting as process 1
db_1  | 2021-05-17T20:01:27.999715Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
db_1  | 2021-05-17T20:01:28.135399Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
db_1  | 2021-05-17T20:01:28.202245Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
db_1  | 2021-05-17T20:01:28.287968Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
db_1  | 2021-05-17T20:01:28.288087Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
db_1  | 2021-05-17T20:01:28.290206Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
db_1  | 2021-05-17T20:01:28.300867Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.25'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.

Migrate the Data using MySQLWorkbench

I have a BUNCH of data (2.2M rows or so) on the original Raspberry Pi.  I want this data in my newly created instance of MySQL.  To get it there I will use the MySQL Workbench migration wizard to move the data from the old to the new instance.

It starts with these nice instructions.

Then I specify the source (the original Raspberry Pi)

The target is specified next.

It then reads the database schema from the source and makes sure that it can talk to the target.

Then it asks me what I want to transfer.  There is only one database schema on the source, the “creekdata” database.

Next it reads the source schema and reverse engineers the tables etc.

Now it asks specifically what you want to transfer.  For my case there are two tables in the creekdata database.

Then it generates the specific mysql commands required to recreate the schema

Gives the option of changing it.

Now it asks you what method you want to use on the target.  I choose to have it do all of the work.

Then it creates the new database and tables.

And you can see that it worked.

Then it asks how I want to copy the data.  I tell it to do all of the work for me.

Then it runs a bulk transfer of the data.

And give me a final report that things worked.  Kick ass.

I can now make a connection to the new database.   And I see my old data back to 2013.

That is it for this article.  In the next article Ill do the Python Shell Script stuff to reconnect my data to the new MySql Server.

The Creek 3.0: Docker Telegraf, Influx, Grafana – Part 1

Summary

The architecture and first steps of a new IoT implementation using PSoC 6, CYW43012 WiFi, AnyCloud MQTT, Raspberry Pi, Python, Influx, Grafana, Telegraf and Docker.  Wow, sounds like a lot.

The Story

For quite some time, I have been wanting to replace my original Elkhorn Creek implementation because. … well …, it is old school and a bit tired.  I started an implementation which I called “The Creek 2.0” which used AWS IoT,  AWS Lambda, and MySQL.  I thought it was interesting to learn about all of the AWS stuff… but I never finished the user interface, nor did I replace the Raspberry Pi.  Also, this solution was going in the old school direction and I wanted to use more open source.

So, this time I am going to go all the way.  Here is the architecture:

There are a bunch of things that I have never used including:

  1. Docker
  2. Mosquito MQTT
  3. Telegraf
  4. Influx DB
  5. Grafana

Which is quite a bit of new stuff.  Almost every time I work on a series like this I do all of the work in advance of writing the first article.  That way I know how things are going to an end and what is going to go wrong.   This way I can fix them in advance of you guys having to suffer with me.  This time, well, not so much, so I am quite sure that there will be some drama.

To this point I have spend a bunch of time with:

  1. Learning Docker
  2. Trying out Influx DB and Grafana
  3. Making Telegraf work

There are still some things which are a bit unknown, including:

  1. I don’t like the Telegraf implementation of the mqtt_consumer, which is going to require me to spend time learning “Go”
  2. I don’t really know how to expose Grafana to the internet safely (is that going to be OK?)
  3. I am considering writing a “Influx Client Library” for PSoC to skip the MQTT?
  4. I am considering using “Influx Line Protocol” and not using MQTT

So over the next few weeks we will see how things evolves.  I also decided to purchase a new Linux box for my house to run the system so I will talk about what I did there.

Alan

Tilt Hydrometer (Part 12) CapSense

Summary

This article updates the user interface to have input as well as output by adding a CapSense GUI.

This series is broken up into the following 12 articles with a few additional possible articles. 

Tilt Hydrometer (Part 1) Overview & Out-of-Box

Tilt Hydrometer (Part 2) Architecture

Tilt Hydrometer (Part 3) Advertising Scanner

Tilt Hydrometer (Part 4) Advertising Packet Error?

Tilt Hydrometer (Part 5) Tilt Simulator & Multi Advertising iBeacons

Tilt Hydrometer (Part 6) Tilt Simulator an Upgrade

Tilt Hydrometer (Part 7) Advertising Database

Tilt Hydrometer (Part 8) Read the Database

Tilt Hydrometer (Part 9) An LCD Display

Tilt Hydrometer (Part 10) The Display State Machine

Tilt Hydrometer (Part 11) Draw the Display Screens

Tilt Hydrometer (Part 12) CapSense

Tilt Hydrometer: LittleFS & SPI Flash (Part ?)

Tilt Hydrometer: WiFi Introducer (Part ?)

Tilt Hydrometer: WebServer (Part ?)

Tilt Hydrometer: Amazon MQTT (Part ?)

Tilt Hydrometer: Printed Circuit Board (Part ?)

You can get the source code from git@github.com:iotexpert/Tilt2.git  This repository has tags for each of the articles which can be accessed with "git checkout part12"  You can find the Tilt Simulator at  git@github.com:iotexpert/TiltSimulator.git.

 

Story

If you look at the development kit you will notice on the right side that there are two CapSense buttons and one slider.  I know that this is going to come as a great shock to those of you who know me, but Im not very patient.  I don’t always want to wait for the system to page through screens every 5 seconds.  So let’s turn on those buttons to do something useful.  But what?  How about the left button will toggle “auto mode” and the right button will go to the next screen.

Board Support Package

We (Cypress/Infineon) created all of the setup stuff you need to make CapSense work on all of our development kits.  If you run “make config” you can look at the configuration of the Board Support Package for this development kit.  Notice that

  1. The CapSense is enabled
  2. The pins are setup for two buttons and a slider.

When you run the CapSense Configurator you can see that there is a slider and two buttons

And they are attached to these pins:

When you run the Library Manager you can also see that the CapSense middleware is already loaded into your project (notice MCU Middleware)

The Firmware

Let’s add the new capsenseManager.h, capsenseManager.c and update main.c.  The capsenseManager.h will just define the task:

#pragma once
void cpm_task();

main.c needs to start the task.

    xTaskCreate(cpm_task, "CapSense Manager", configMINIMAL_STACK_SIZE*2,0 /* args */ ,0 /* priority */, 0);

All of the action takes place in capsenseManager.c.  In the file there are really only two things that are even mildly complicated.  The CapSense block is a combination of a hardware block plus some firmware plus some middleware.  The hardware block does the CapSense to digital conversion then triggers an interrupt.  The interrupt firmware is responsible for managing the hardware block including sequencing the measurements, running the baseline etc.  When you trigger a scan, there is a back and forth between the hardware block and the firmware that must happen in an ISR.  Finally when things are done, you can ask for a callback.

The flow looks like this

  1. Initialize the hardware
  2. Initialize the Interrupt Service Routine
  3. Ask for a Callback
  4. Enable the CapSense block
  5. Start a scan
  6. Wait for the callback
  7. Process the results
  8. Start a scan (repeat)

Initialize the CapSense (steps 1-4)

To get this going you need to:

  1. Define the task
  2. Initialize a semaphore (which you will use in the callback)
  3. Initialize the hardware block
  4. Initialize the interrupt
  5. Register the callback
  6. Enable the CapSense block
void cpm_task()
{
    cpm_semaphore = xSemaphoreCreateCounting(10,0);

    static const cy_stc_sysint_t CapSense_ISR_cfg =
    {
        .intrSrc = csd_interrupt_IRQn, /* Interrupt source is the CSD interrupt */
        .intrPriority = 7u,            /* Interrupt priority is 7 */
    };

    Cy_CapSense_Init(&cy_capsense_context);
        
    Cy_SysInt_Init(&CapSense_ISR_cfg, &cpm_isr);
    NVIC_ClearPendingIRQ(CapSense_ISR_cfg.intrSrc);
    NVIC_EnableIRQ(CapSense_ISR_cfg.intrSrc);

    Cy_CapSense_RegisterCallback	(CY_CAPSENSE_END_OF_SCAN_E,cpm_callback, &cy_capsense_context); 
    Cy_CapSense_Enable (&cy_capsense_context);

The ISR & CallBack

The ISR is trivial.  All it does is run our interrupt handler.

The callback just gives a semaphore which has held the task in suspension while the CapSense is running.

static void cpm_isr(void)
{
    Cy_CapSense_InterruptHandler(CYBSP_CSD_HW, &cy_capsense_context);
}

static void cpm_callback(cy_stc_active_scan_sns_t *ptrActiveScan)
{
    xSemaphoreGiveFromISR(cpm_semaphore,portMAX_DELAY);
}

Main Task Loop

In the main loop I

  1. Scan all of the widgets… then wait until the scan is done by holding on the semaphore.
  2. After the scan is done I need to run all of the Cypress code which manages the data.
  3. Then I find out the state of the two buttons.
  4. Based on the state I call either the toggle auto mode or next screen command

Notice that I wait for 20ms after I get this done.  What this does is give me a GUI update rate of about 50hz.  Probably 10hz would be fine.

  int button0Prev = 0;
    int button1Prev = 0;
    int button0Curr = 0;
    int button1Curr = 0;
    while(1)
    {
        Cy_CapSense_ScanAllWidgets (&cy_capsense_context);
        xSemaphoreTake(cpm_semaphore,portMAX_DELAY);
        Cy_CapSense_ProcessAllWidgets(&cy_capsense_context);
        button0Curr = Cy_CapSense_IsWidgetActive(CY_CAPSENSE_BUTTON0_WDGT_ID,&cy_capsense_context);
        button1Curr = Cy_CapSense_IsWidgetActive(CY_CAPSENSE_BUTTON1_WDGT_ID,&cy_capsense_context);

        if(button0Curr != button0Prev && button0Curr == 1)
        {
            dm_submitAutoCmd();
        }
        if(button1Curr != button1Prev && button1Curr == 1)
        {
            dm_submitNextScreenCmd();
        }

        button0Prev = button0Curr;
        button1Prev = button1Curr;

        vTaskDelay(20); // 50 Hz Update Rate

    }

capSenseManager.c

Here is the whole file.

#include <stdio.h>

#include "cycfg_capsense.h"

#include "FreeRTOS.h"
#include "task.h"
#include "semphr.h"

#include "displayManager.h"

static SemaphoreHandle_t cpm_semaphore;

static void cpm_isr(void)
{
    Cy_CapSense_InterruptHandler(CYBSP_CSD_HW, &cy_capsense_context);
}

static void cpm_callback(cy_stc_active_scan_sns_t *ptrActiveScan)
{
    xSemaphoreGiveFromISR(cpm_semaphore,portMAX_DELAY);
}

void cpm_task()
{
    cpm_semaphore = xSemaphoreCreateCounting(10,0);

    static const cy_stc_sysint_t CapSense_ISR_cfg =
    {
        .intrSrc = csd_interrupt_IRQn, /* Interrupt source is the CSD interrupt */
        .intrPriority = 7u,            /* Interrupt priority is 7 */
    };

    Cy_CapSense_Init(&cy_capsense_context);
        
    Cy_SysInt_Init(&CapSense_ISR_cfg, &cpm_isr);
    NVIC_ClearPendingIRQ(CapSense_ISR_cfg.intrSrc);
    NVIC_EnableIRQ(CapSense_ISR_cfg.intrSrc);

    Cy_CapSense_RegisterCallback	(CY_CAPSENSE_END_OF_SCAN_E,cpm_callback, &cy_capsense_context); 
    Cy_CapSense_Enable (&cy_capsense_context);

    int button0Prev = 0;
    int button1Prev = 0;
    int button0Curr = 0;
    int button1Curr = 0;
    while(1)
    {
        Cy_CapSense_ScanAllWidgets (&cy_capsense_context);
        xSemaphoreTake(cpm_semaphore,portMAX_DELAY);
        Cy_CapSense_ProcessAllWidgets(&cy_capsense_context);
        button0Curr = Cy_CapSense_IsWidgetActive(CY_CAPSENSE_BUTTON0_WDGT_ID,&cy_capsense_context);
        button1Curr = Cy_CapSense_IsWidgetActive(CY_CAPSENSE_BUTTON1_WDGT_ID,&cy_capsense_context);

        if(button0Curr != button0Prev && button0Curr == 1)
        {
            dm_submitAutoCmd();
        }
        if(button1Curr != button1Prev && button1Curr == 1)
        {
            dm_submitNextScreenCmd();
        }

        button0Prev = button0Curr;
        button1Prev = button1Curr;

        vTaskDelay(20); // 50 Hz Update Rate

    }
}

 

Tilt Hydrometer (Part 11) Draw the Display Screens

Summary

This article updates the display system in my Tilt Hydrometer project to actually display useful information.

This series is broken up into the following 12 articles with a few additional possible articles. 

Tilt Hydrometer (Part 1) Overview & Out-of-Box

Tilt Hydrometer (Part 2) Architecture

Tilt Hydrometer (Part 3) Advertising Scanner

Tilt Hydrometer (Part 4) Advertising Packet Error?

Tilt Hydrometer (Part 5) Tilt Simulator & Multi Advertising iBeacons

Tilt Hydrometer (Part 6) Tilt Simulator an Upgrade

Tilt Hydrometer (Part 7) Advertising Database

Tilt Hydrometer (Part 8) Read the Database

Tilt Hydrometer (Part 9) An LCD Display

Tilt Hydrometer (Part 10) The Display State Machine

Tilt Hydrometer (Part 11) Draw the Display Screens

Tilt Hydrometer (Part 12) CapSense

Tilt Hydrometer: LittleFS & SPI Flash (Part ?)

Tilt Hydrometer: WiFi Introducer (Part ?)

Tilt Hydrometer: WebServer (Part ?)

Tilt Hydrometer: Amazon MQTT (Part ?)

Tilt Hydrometer: Printed Circuit Board (Part ?)

You can get the source code from git@github.com:iotexpert/Tilt2.git  This repository has tags for each of the articles which can be accessed with "git checkout part12"  You can find the Tilt Simulator at  git@github.com:iotexpert/TiltSimulator.git.

 

This article will continue to edit the display manager task

Story

In the last article I built all of the infrastructure to make the state machine work.  Now it is time to draw some actual graphics and get this puppy displaying some data.  Remember that I want three screens

  1. The Splash Screen with the IoT Expert Logo

2. The Table Screen with a table of Gravity and Temperature for the 8 Tilts

3. A detail screen.  One per Tilt with more information drawn in the same color as the Tilt.

From the last post you might remember that you need to write 4 functions to implement a new screen

  1. precheck – returns true if it is legal to come to that screen
  2. init – draw the initial data (like the table outline)
  3. update – to update the data on the screen
  4. sequence – up move to the next subscreen e.g. purple –> red

Splash Screen

First the splash screen.  The pre-check function returns true because you are always allowed to “go” to this screen.  The initialization function draws all of the screen (as it is static).  Basically sets the background to white, clears the screen (which actually turns it white) then draws the bitmap.  There is nothing to do in the update or the sequence functions.

////////////////////////////////////////////////////////////////////////////////
//
// Splash
// 
////////////////////////////////////////////////////////////////////////////////
static bool dm_displayScreenSplashPre()
{
    return true;
}

static void dm_displayScreenSplashInit()
{
    GUI_SetBkColor(GUI_WHITE);
    GUI_Clear();
    GUI_DrawBitmap(&bmIOTexpert_Logo_Vertical320x240,0,17);

}
static void dm_displayScreenSplashUpdate()
{
}

static bool dm_displayScreenSplashSeq()
{
    return true;
}

Table Screen

The table screen precheck and sequence functions say “true”, meaning it is always OK to come to this screen and it is always OK to leave this screen.

static bool dm_displayScreenTablePre()
{
    return true;
}

static bool dm_displayScreenTableSeq()
{
    return true;
}

The initialization function draws the header and the two lines.

First of all it sets the location of the graphics with #defines.  The actual function goes on to use that information and draw the picture.  Self explanatory.

#define TABLE_FONT (GUI_FONT_24B_ASCII)
#define TABLE_BGCOLOR (GUI_BLACK)
#define TABLE_HEAD_BGCOLOR (GUI_WHITE)

#define TABLE_NAME_LEFT_X (0)
#define TABLE_GRAV_LEFT_X (120)
#define TABLE_TEMP_LEFT_X (220)

#define TABLE_NAME_RIGHT_X (119)
#define TABLE_GRAV_RIGHT_X (219)
#define TABLE_TEMP_RIGHT_X (319)

#define TABLE_NAME_CENTER_X (TABLE_NAME_LEFT_X+(TABLE_NAME_RIGHT_X-TABLE_NAME_LEFT_X)/2)
#define TABLE_GRAV_CENTER_X (TABLE_GRAV_LEFT_X+(TABLE_GRAV_RIGHT_X-TABLE_GRAV_LEFT_X)/2)
#define TABLE_TEMP_CENTER_X (TABLE_TEMP_LEFT_X+(TABLE_TEMP_RIGHT_X-TABLE_TEMP_LEFT_X)/2)

static void dm_displayScreenTableInit()
{
    GUI_SetColor(TABLE_HEAD_BGCOLOR);
    GUI_SetBkColor(TABLE_BGCOLOR);
    GUI_SetFont(TABLE_FONT);
    GUI_Clear();

    GUI_FillRect(0,0,320,GUI_GetFontSizeY()+ TOP_MARGIN);

    GUI_SetTextMode(GUI_TM_REV);
    GUI_SetTextAlign(GUI_TA_CENTER);
    GUI_DispStringHCenterAt("Name",TABLE_NAME_CENTER_X,TOP_MARGIN);
    GUI_DispStringHCenterAt("Gravity",TABLE_GRAV_CENTER_X,TOP_MARGIN);
    GUI_DispStringHCenterAt("Temp",TABLE_TEMP_CENTER_X,TOP_MARGIN);

    GUI_DrawLine(TABLE_NAME_RIGHT_X,0,TABLE_NAME_RIGHT_X,240);
    GUI_DrawLine(TABLE_GRAV_RIGHT_X,0,TABLE_GRAV_RIGHT_X,240);
}

The update will:

  1. set the graphics configuration
  2. iterate through the list of Tilts
  3. If it is active then it will get the data
  4. sprintf it into a buffer than display it.
  5. If it is not active then display a “–“
static void dm_displayScreenTableUpdate()
{

    uint32_t activeTilts =tdm_getActiveTiltMask();

    char buff[64];
    
    GUI_SetColor(TABLE_HEAD_BGCOLOR);
    GUI_SetBkColor(TABLE_BGCOLOR);
    
    GUI_SetFont(TABLE_FONT);
    GUI_SetTextMode(GUI_TEXTMODE_NORMAL);
    GUI_SetTextAlign(GUI_TA_CENTER);

    int row;
    for(int i=0;i<tdm_getNumTilt();i++)
    {
        row = i+1;

        GUI_SetColor(tdm_colorGUI(i));
        GUI_DispStringHCenterAt(tdm_getColorString(i), TABLE_NAME_CENTER_X, ROW_Y(row));

        if(1<<i & activeTilts)
        {

            tdm_tiltData_t *response = tdm_getTiltData(i);

            sprintf(buff,"%1.3f",response->gravity);
            GUI_DispStringHCenterAt(buff, TABLE_GRAV_CENTER_X, ROW_Y(row));
            sprintf(buff,"%02d",response->temperature);
            GUI_DispStringHCenterAt(buff, TABLE_TEMP_CENTER_X, ROW_Y(row));
                        
            free(response);
        }
        else
        {
            GUI_DispStringHCenterAt("-----", TABLE_GRAV_CENTER_X, ROW_Y(row));
            GUI_DispStringHCenterAt("--", TABLE_TEMP_CENTER_X, ROW_Y(row));

        }
    }  
}

The only little bit of trickiness is that I defined a macro to calculate the y position on the screen.  Specifically I divided the screen into “rows” starting at 0 based on the height of the font plus the margin between the rows plus whatever the offset at the top of the screen (the top margin)

#define TOP_MARGIN (4)
#define LINE_MARGIN (2)
#define ROW_Y(row) (TOP_MARGIN + (row)*(LINE_MARGIN+GUI_GetFontSizeY()))

Single Screen

The single screen is a bit more interesting.  First of all you are only allowed to go to this screen if there is at least one Tilt that is active.  The next thing is that I want the first time you go to this screen to go to the first active tilt.

static tdm_tiltHandle_t currentSingle = 0xFF;

static bool dm_displaySinglePre()
{
    uint32_t activeTilts =tdm_getActiveTiltMask();
    if(activeTilts == 0)
        return false;

    for(int i=0;i<tdm_getNumTilt();i++)
    {
        if(1<<i & activeTilts)
        {
            currentSingle = i;
            break;
        }
    }

    return true;
}

The initialization function

  1. Clears the screen
  2. Asks for the color of the current Tilt (so that all of the text is drawn using that color)
  3. Draws the header
  4. Draws the labels
static void dm_displaySingleInit()
{
    GUI_SetBkColor(GUI_BLACK);
    GUI_SetFont(GUI_FONT_32B_ASCII);
    GUI_Clear();

    GUI_SetColor(tdm_colorGUI(currentSingle));

    GUI_FillRect(0,0,320,ROW_Y(1));
    
    GUI_SetTextAlign(GUI_TA_LEFT);
    GUI_SetTextMode(GUI_TM_REV);
    GUI_DispStringHCenterAt(tdm_getColorString(currentSingle), CENTER_X,ROW_Y(0) );

    GUI_SetTextMode(GUI_TM_NORMAL);
    GUI_SetTextAlign(GUI_TA_RIGHT | GUI_TA_VCENTER);

    GUI_DispStringAt("Gravity: ", SINGLE_LABEL_X,ROW_Y(SINGLE_GRAV_ROW) );
    GUI_SetTextAlign(GUI_TA_RIGHT | GUI_TA_VCENTER);
    GUI_DispStringAt("Temp: ",    SINGLE_LABEL_X,ROW_Y(SINGLE_TEMP_ROW) );
    GUI_SetTextAlign(GUI_TA_RIGHT | GUI_TA_VCENTER);
    GUI_DispStringAt("TxPower: ", SINGLE_LABEL_X,ROW_Y(SINGLE_TXPOWER_ROW) );
    GUI_SetTextAlign(GUI_TA_RIGHT | GUI_TA_VCENTER);
    GUI_DispStringAt("RSSI: ",    SINGLE_LABEL_X,ROW_Y(SINGLE_RSSI_ROW) );
    GUI_SetTextAlign(GUI_TA_RIGHT | GUI_TA_VCENTER);
    GUI_DispStringAt("Time: ",    SINGLE_LABEL_X,ROW_Y(SINGLE_TIME_ROW) );

}

The update function basically sets the font and color (just to make sure) then it gets the data, formats it into a string and displays it.  When I wrote this function originally I wasn’t sure if this would ever be called in a  situation where there is no data, so I handled that with the “—-“, but I don’t think that branch is actually ever used.

static void dm_displaySingleUpdate()
{

    uint32_t activeTilts =tdm_getActiveTiltMask();

    char gravString[10];
    char tempString[10];
    char txPowerString[10];
    char rssiString[10];
    char timeString[64];
       
    GUI_SetBkColor(GUI_BLACK);
    GUI_SetFont(GUI_FONT_32B_ASCII);
    GUI_SetColor(tdm_colorGUI(currentSingle));
    
    if(1<<currentSingle & activeTilts)
    {
        tdm_tiltData_t *response = tdm_getTiltData(currentSingle);
        sprintf(gravString,"%1.3f",response->gravity);
        sprintf(tempString,"%02d",response->temperature);
        sprintf(txPowerString,"%d",response->txPower);
        sprintf(rssiString,"%2d",response->rssi);

        int seconds = (xTaskGetTickCount()/1000- response->time);
        int days = seconds/(24*60*60);
        seconds = seconds - days*(24*60*60);
        int hours = seconds/(60*60);
        seconds = seconds - hours*(60*60);
        int minutes = seconds/60;
        seconds = seconds - (minutes * 60);
        
        sprintf(timeString,"%02d:%02d:%02d:%02d",days,hours,minutes,seconds);                       
        free(response);
    }

    else
    {
        sprintf(gravString,"-----");
        sprintf(tempString,"--");
        sprintf(txPowerString,"---");
        sprintf(rssiString,"--");
        sprintf(timeString,"---");                       
    }

    GUI_DispStringAtCEOL(gravString, SINGLE_VALUE_X,ROW_Y(SINGLE_GRAV_ROW) );
    GUI_SetTextAlign(GUI_TA_LEFT | GUI_TA_VCENTER);
    GUI_DispStringAtCEOL(tempString,    SINGLE_VALUE_X,ROW_Y(SINGLE_TEMP_ROW) );
    GUI_SetTextAlign(GUI_TA_LEFT | GUI_TA_VCENTER);
    GUI_DispStringAtCEOL(txPowerString, SINGLE_VALUE_X,ROW_Y(SINGLE_TXPOWER_ROW) );
    GUI_SetTextAlign(GUI_TA_LEFT | GUI_TA_VCENTER);
    GUI_DispStringAtCEOL(rssiString,    SINGLE_VALUE_X,ROW_Y(SINGLE_RSSI_ROW) );
    GUI_SetTextAlign(GUI_TA_LEFT | GUI_TA_VCENTER);
    GUI_DispStringAtCEOL(timeString,    SINGLE_VALUE_X,ROW_Y(SINGLE_TIME_ROW) );
}

The last bit of code is the sequence which will either

  1. Go to the next screen by returning True (if there are no active Tilts)
  2. Move to the next Tilt (if there are more active)
  3. Go to the next screen by returning True
static bool dm_displaySingleSeq()
{
    uint32_t activeTilts =tdm_getActiveTiltMask();
    if(activeTilts == 0)
        return true;

    for(int i=currentSingle+1;i<tdm_getNumTilt();i++)
    {
        if(1<<i & activeTilts)
        {
            currentSingle = i;
            dm_displaySingleInit();
            return false;
        }
    }
    return true;
}

That’s it for the display.  In the next article Ill add the User Input of the GUI by adding CapSense.

Tilt Hydrometer (Part 10) The Display State Machine

Summary

This article updates the display system in my Tilt Hydrometer project to include the state machine apparatus to move between screens.

This series is broken up into the following 12 articles with a few additional possible articles. 

Tilt Hydrometer (Part 1) Overview & Out-of-Box

Tilt Hydrometer (Part 2) Architecture

Tilt Hydrometer (Part 3) Advertising Scanner

Tilt Hydrometer (Part 4) Advertising Packet Error?

Tilt Hydrometer (Part 5) Tilt Simulator & Multi Advertising iBeacons

Tilt Hydrometer (Part 6) Tilt Simulator an Upgrade

Tilt Hydrometer (Part 7) Advertising Database

Tilt Hydrometer (Part 8) Read the Database

Tilt Hydrometer (Part 9) An LCD Display

Tilt Hydrometer (Part 10) The Display State Machine

Tilt Hydrometer (Part 11) Draw the Display Screens

Tilt Hydrometer (Part 12) CapSense

Tilt Hydrometer: LittleFS & SPI Flash (Part ?)

Tilt Hydrometer: WiFi Introducer (Part ?)

Tilt Hydrometer: WebServer (Part ?)

Tilt Hydrometer: Amazon MQTT (Part ?)

Tilt Hydrometer: Printed Circuit Board (Part ?)

You can get the source code from git@github.com:iotexpert/Tilt2.git  This repository has tags for each of the articles which can be accessed with "git checkout part12"  You can find the Tilt Simulator at  git@github.com:iotexpert/TiltSimulator.git.

 

We will continue to edit the Display Manager in this article:

Story

Things are a little unfair as I already know the end of this story because I wrote this code a few months ago.  Recently, as I came back to write these articles I looked at the code, actually Monday.  The code was pretty complicated and I have been really busy so I set it aside as I wasn’t sure how to explain it.  Then I looked again on Tuesday and contemplated re-writing it… then again on Wednesday then … and finally Saturday when I decided that what I had done originally was correct.  That means I just need to explain it.

My system is going to look work like this:

  1. A splash screen with the IoT Expert Logo
  2. A table of data screen with one row per tilt
  3. A series of details screens, one per tilt
  4. The ability to “skip” detail screens if there is no data
  5. An automated UI that moved through the screens every 5000ms
  6. Support for a manual UI so that the ntshell and the CapSense interface could send it commands to move to specific screens

Here is the picture:

Add Color to the Tilt Data Manager

Before I jump into the GUI, I need to add some color information to the database of Tilts.  This is a little bit of a smearing the line between the database and the display systems, but I felt that having all of the information about the Tilts in one place was better than having a split.  In order for the display manager to get the color information I add a new function to the tiltDataManager.h to return the specific GUI_COLOR (which is an emWin thing) for that specific Tilt handle.

GUI_COLOR tdm_colorGUI(tdm_tiltHandle_t handle)

Then I need to update the tiltDataManager. c to have the color in the database.  Notice that the emWin library doesn’t have purpose or pink so I create those colors.

typedef struct  {
    char *colorName;
    GUI_COLOR color;
    uint8_t uuid[20];
    tdm_tiltData_t *data;
    int numDataPoints;
    int numDataSeen;
} tilt_t;


#define IBEACON_HEADER 0x4C,0x00,0x02,0x15
#define GUI_PINK GUI_MAKE_COLOR(0x00CCCCFF)
#define GUI_PURPLE GUI_MAKE_COLOR(0x00800080)

static tilt_t tiltDB [] =
{
    {"Red",    GUI_RED,    {IBEACON_HEADER,0xA4,0x95,0xBB,0x10,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Green" , GUI_GREEN,  {IBEACON_HEADER,0xA4,0x95,0xBB,0x20,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Black" , GUI_GRAY,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x30,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Purple", GUI_PURPLE, {IBEACON_HEADER,0xA4,0x95,0xBB,0x40,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Orange", GUI_ORANGE, {IBEACON_HEADER,0xA4,0x95,0xBB,0x50,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Blue"  , GUI_BLUE,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x60,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Yellow", GUI_YELLOW, {IBEACON_HEADER,0xA4,0x95,0xBB,0x70,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
    {"Pink"  , GUI_PINK,   {IBEACON_HEADER,0xA4,0x95,0xBB,0x80,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE},0,0,0},
};
#define NUM_TILT (sizeof(tiltDB)/sizeof(tilt_t))

Then I add the actual function.

GUI_COLOR tdm_colorGUI(tdm_tiltHandle_t handle)
{
    return tiltDB[handle].color;
}

Display Manager State Machine

The picture above looks like a state machine, which I suppose makes sense as it is a state machine.  The parts of the state machine are

  1. Initialize the screen
  2. Update the data on the screen
  3. Go to the next screen
  4. Sequence the “subscreen” i.e. from the purple screen to the red screen
  5. Precheck (can you enter the screen)

To support this I created a structure of function pointers:

typedef struct {
    bool (*precheck)(void);   // return true if you can come to this screen
    void (*init)(void);       // draw the initial stuff
    void (*update)(void);     // update the data
    bool (*sequence)(void);   // sequence the data .. return true if you should go to the next screen
    dm_screenName_t next;
} dm_screenMgmt_t;

This means that each “screen” needs functions in a table that have:

  1. precheck – returns true if it is legal to come to that screen
  2. init – draw the initial data (like the table outline)
  3. update – to update the data on the screen
  4. sequence – up move to the next subscreen e.g. purple –> red

But what is the “next” member in the structure?  This is just an index into the state machine table that tells what is the next entry to go too.  I wish that there was a better way to do this in C as the enumerated value is critically mapped to the place in the array of screens (see below)

typedef enum {
    SPLASH,
    TABLE,
    SINGLE,
} dm_screenName_t;

With all of this setup I can now make a table to represent the states of my screens like this (more on the functions a bit later in this article).

dm_screenMgmt_t screenList[] = {
    {dm_displayScreenSplashPre, dm_displayScreenSplashInit, dm_displayScreenSplashUpdate, dm_displayScreenSplashSeq, TABLE},
    {dm_displayScreenTablePre , dm_displayScreenTableInit, dm_displayScreenTableUpdate, dm_displayScreenTableSeq, SINGLE},
    {dm_displaySinglePre, dm_displaySingleInit, dm_displaySingleUpdate, dm_displaySingleSeq, TABLE},
};

With that in place I can now create the code that actually runs the state machine, dm_nextScreen.  This function

  1. Runs the “sequence” function that will either move to the next subscreen OR it will return true (meaning go to the next screen)
  2. If the precheck returns true then it is legal to jump to this screen
  3. Jump to the next screen
  4. Finally update the data on the current screen
static void dm_nextScreen()
{
    if((*screenList[dm_currentScreen].sequence)())
    {
        if((*screenList[screenList[dm_currentScreen].next].precheck)())
        {
            dm_currentScreen = screenList[dm_currentScreen].next;
            (*screenList[dm_currentScreen].init)();
        }
    }
    (*screenList[dm_currentScreen].update)();
}

The Display Manager Task

The display manager task is now a bit tricky as well.  First of all I have a boolean variable called “autoRotate”.  When this variable is true it means that the screens should automatically switch to the next screen every 5 seconds.

The next part of the code initializes a command queue (so that other tasks can send a next screen or enable/disable of autorotate or a jump straight to the table.

The dm_currentScreen is global to this file and keep track of which screen you are currently on, which is what state the state machine is in.

The next part of the code waits for a message OR a timeout.  The timeout happens after 5000ms (5s) and tell the system to either go to the next screen, or update the data on the screen.

void dm_task(void *arg)
{
    dm_cmdMsg_t msg;
    bool autoRotate=true;
    dm_cmdQueue = xQueueCreate(10,sizeof(dm_cmdMsg_t));

    /* Initialize the display controller */
    mtb_st7789v_init8(&tft_pins);
    GUI_Init();

    dm_currentScreen = SPLASH;

    dm_displayScreenSplashInit();
    dm_displayScreenSplashUpdate();

    for(;;)
    {
        if(xQueueReceive(dm_cmdQueue,&msg,5000) == pdPASS) // Got a command
        {
            switch(msg.cmd)
            {
                case SCREEN_NEXT:
                dm_nextScreen();
                break;
                case SCREEN_AUTO:
                autoRotate = ! autoRotate;
                printf("AutoRotate =%s\n",autoRotate?"True":"False");
                break;
                case SCREEN_TABLE:
                dm_currentScreen = TABLE;
                (*screenList[dm_currentScreen].init)();
                (*screenList[dm_currentScreen].update)();
                break;

            }

        }
        else // otherwise just update the screen
        {
            if(autoRotate)
                dm_nextScreen();
            else
                (*screenList[dm_currentScreen].update)();
        }
    }
}

With the infrastructure in place, in the next article I actually draw the screens.

Tilt Hydrometer (Part 9) An LCD Display

Summary

This article updates the Tilt Hydrometer project to include an LCD display using the Segger emWin library.  This includes a discussion of a “new” library in the suite of IoT Expert awesome sauce.  Specifically the IoT Expert Graphics library.

This series is broken up into the following 12 articles with a few additional possible articles. 

Tilt Hydrometer (Part 1) Overview & Out-of-Box

Tilt Hydrometer (Part 2) Architecture

Tilt Hydrometer (Part 3) Advertising Scanner

Tilt Hydrometer (Part 4) Advertising Packet Error?

Tilt Hydrometer (Part 5) Tilt Simulator & Multi Advertising iBeacons

Tilt Hydrometer (Part 6) Tilt Simulator an Upgrade

Tilt Hydrometer (Part 7) Advertising Database

Tilt Hydrometer (Part 8) Read the Database

Tilt Hydrometer (Part 9) An LCD Display

Tilt Hydrometer (Part 10) The Display State Machine

Tilt Hydrometer (Part 11) Draw the Display Screens

Tilt Hydrometer (Part 12) CapSense

Tilt Hydrometer: LittleFS & SPI Flash (Part ?)

Tilt Hydrometer: WiFi Introducer (Part ?)

Tilt Hydrometer: WebServer (Part ?)

Tilt Hydrometer: Amazon MQTT (Part ?)

Tilt Hydrometer: Printed Circuit Board (Part ?)

You can get the source code from git@github.com:iotexpert/Tilt2.git  This repository has tags for each of the articles which can be accessed with "git checkout part12"  You can find the Tilt Simulator at  git@github.com:iotexpert/TiltSimulator.git.

 

Here is the architecture with the blocks we are editing marked green.

Story

I knew that I wanted to use the TFT Display so that I could have this Tilt Hydrometer system sitting in my “brewery”.  Notice in the picture below you can see that the screen is currently displaying the data from the “black” tilt.  Also notice that you can see two fermenters in the background in the brewery.  Who needs a guest bathroom in the era of Covid 🙂

Also notice that there is a Raspberry Pi (more on that in a future article).  And for all of you married people out there, yes I have a very frustrated but forgiving wife.

Add Display Libraries & Fix the Makefile

The first step in building this is to add the new libraries.  Specifically the display-tft-st7789v library which knows how to talk to the lcd and the emWin library.  AND the new IoT Expert Graphics Library.

In order to use the emWin library you need to fix the Makefile to include the emWin component.  Notice that I chose OS  meaning an RTOS and NTS meaning no touch screen.

COMPONENTS=FREERTOS WICED_BLE EMWIN_OSNTS

IoT Expert Graphics Library

I wanted to have a splash screen on this project with my Logo.  So I started digging around on my computer to find the graphics, which I found in the file IOTexpert_Logo_Vertical.png.  Unfortunately this is a 1091×739 portable network graphics file, not exactly what emWin wants to draw with.  To fix this I opened it in GIMP

Then I resized it to my screen size (320×240)

Now it looks like this:

Then I saved it as a PNG.  Still doesn’t work with emWin.  But, if you look in the emWin library you will notice that there is a new directory called “Tool”

This directory contains the Segger tools for supporting their graphics library.  The one that I want is called “Bin2C.exe” (unfortunately it is a Windows only tool).  I run it and open the logo.

Then I click save and pick “C” bitmap file (*.c)

For my display I am using 16-bit color, also known as High Color with Alpha 565.

The tool nicely makes me a file with the right stuff in it.  First, the array of unsigned char (representing the bitmap) and the GUI_BITMAP structure.

Unfortunately it didn’t create a header file which I do like this (notice I renamed the data)

#pragma once
#include <stdlib.h>

#include "GUI.h"

extern GUI_CONST_STORAGE GUI_BITMAP bmIOTexpert_Logo_Vertical320x240;

And, if you have been paying attention you might have noticed that I created a new IoT Expert Library – Graphics that contains these new files. (I have explained this process several times in the past)

TFT Pins Discussion

I am planning on making a custom circuit board for this whole project.  So, I didn’t want to use the whole CY8CKIT-028-TFT library.  In order to start the TFT you need to call the function “mtb_st7789v()” with an argument of type “mtb_st7789v_pins_t”.  When Cypress created the BSP for the TFT we defined a function like this in the header file.

/**
 * Initializes GPIOs for the software i8080 interface.
 * @return CY_RSLT_SUCCESS if successfully initialized, else an error about
 * what went wrong
 */
static inline cy_rslt_t cy_tft_io_init(void)
{
    static const mtb_st7789v_pins_t PINS =
    {
        .db08 = CY8CKIT_028_TFT_PIN_DISPLAY_DB8,
        .db09 = CY8CKIT_028_TFT_PIN_DISPLAY_DB9,
        .db10 = CY8CKIT_028_TFT_PIN_DISPLAY_DB10,
        .db11 = CY8CKIT_028_TFT_PIN_DISPLAY_DB11,
        .db12 = CY8CKIT_028_TFT_PIN_DISPLAY_DB12,
        .db13 = CY8CKIT_028_TFT_PIN_DISPLAY_DB13,
        .db14 = CY8CKIT_028_TFT_PIN_DISPLAY_DB14,
        .db15 = CY8CKIT_028_TFT_PIN_DISPLAY_DB15,
        .nrd = CY8CKIT_028_TFT_PIN_DISPLAY_NRD,
        .nwr = CY8CKIT_028_TFT_PIN_DISPLAY_NWR,
        .dc = CY8CKIT_028_TFT_PIN_DISPLAY_DC,
        .rst = CY8CKIT_028_TFT_PIN_DISPLAY_RST,
    };
    return mtb_st7789v_init8(&PINS);
}

Curiously we also made a static definition inside of the .c file.

static const mtb_st7789v_pins_t tft_pins =
{
    .db08 = CY8CKIT_028_TFT_PIN_DISPLAY_DB8,
    .db09 = CY8CKIT_028_TFT_PIN_DISPLAY_DB9,
    .db10 = CY8CKIT_028_TFT_PIN_DISPLAY_DB10,
    .db11 = CY8CKIT_028_TFT_PIN_DISPLAY_DB11,
    .db12 = CY8CKIT_028_TFT_PIN_DISPLAY_DB12,
    .db13 = CY8CKIT_028_TFT_PIN_DISPLAY_DB13,
    .db14 = CY8CKIT_028_TFT_PIN_DISPLAY_DB14,
    .db15 = CY8CKIT_028_TFT_PIN_DISPLAY_DB15,
    .nrd = CY8CKIT_028_TFT_PIN_DISPLAY_NRD,
    .nwr = CY8CKIT_028_TFT_PIN_DISPLAY_NWR,
    .dc = CY8CKIT_028_TFT_PIN_DISPLAY_DC,
    .rst = CY8CKIT_028_TFT_PIN_DISPLAY_RST,
};

What I know is that I need those pin definitions, so I will copy them into my project (a bit later in this article)

Make a New Display Task

To get this going, make a file called displayManager.h to contain the public definition of the task.

#pragma once

void dm_task(void *arg);

Then add a file displayManager.c.  This file will be the focus of the rest of this article.  Before we get too far down the road building all of the functionality of the display, first lets setup a test task to verify the functionality of the libraries and the screen.  As I talked about earlier, define the pins for the driver.  These the pin names come out of the BSP and “should” work for all of the Cypress development kits (that have Arduino headers).

#include <stdio.h>
#include <stdlib.h>

#include "GUI.h"
#include "mtb_st7789v.h"
#include "cybsp.h"

#include "FreeRTOS.h"
#include "task.h"

#include "displayManager.h"

#include "IOTexpert_Logo_Vertical320x240.h"

const mtb_st7789v_pins_t tft_pins =
{
    .db08 = (CYBSP_J2_2),
    .db09 = (CYBSP_J2_4),
    .db10 = (CYBSP_J2_6),
    .db11 = (CYBSP_J2_10),
    .db12 = (CYBSP_J2_12),
    .db13 = (CYBSP_D7),
    .db14 = (CYBSP_D8),
    .db15 = (CYBSP_D9),
    .nrd  = (CYBSP_D10),
    .nwr  = (CYBSP_D11),
    .dc   = (CYBSP_D12),
    .rst  = (CYBSP_D13)
};

Then make a simple task that

  1. Initializes the driver
  2. Initialize emWin
  3. Sets the background color
  4. Clear the screen
  5. Draw the IoT Expert Logo
void dm_task(void *arg)
{
    /* Initialize the display controller */
    mtb_st7789v_init8(&tft_pins);
    GUI_Init();
    GUI_SetBkColor(GUI_WHITE);
    GUI_Clear();
    GUI_DrawBitmap(&bmIOTexpert_Logo_Vertical320x240,0,11);

    while(1)
        vTaskDelay(portMAX_DELAY);
}

Test

When I program this project I get a nice white background screen with my beautiful logo on it. Sweet.  In the next article I’ll do some work on displaying useful information on the screen.

 

Tilt Hydrometer (Part 8) Read the Database

Summary

This article expands the functionality of the Tilt Database by adding functions to get the status of Tilts from other threads in the system.  It demonstrates the FreeRTOS queue’s to provide a generic mechanism for communication between threads.

This series is broken up into the following 12 articles with a few additional possible articles. 

Tilt Hydrometer (Part 1) Overview & Out-of-Box

Tilt Hydrometer (Part 2) Architecture

Tilt Hydrometer (Part 3) Advertising Scanner

Tilt Hydrometer (Part 4) Advertising Packet Error?

Tilt Hydrometer (Part 5) Tilt Simulator & Multi Advertising iBeacons

Tilt Hydrometer (Part 6) Tilt Simulator an Upgrade

Tilt Hydrometer (Part 7) Advertising Database

Tilt Hydrometer (Part 8) Read the Database

Tilt Hydrometer (Part 9) An LCD Display

Tilt Hydrometer (Part 10) The Display State Machine

Tilt Hydrometer (Part 11) Draw the Display Screens

Tilt Hydrometer (Part 12) CapSense

Tilt Hydrometer: LittleFS & SPI Flash (Part ?)

Tilt Hydrometer: WiFi Introducer (Part ?)

Tilt Hydrometer: WebServer (Part ?)

Tilt Hydrometer: Amazon MQTT (Part ?)

Tilt Hydrometer: Printed Circuit Board (Part ?)

You can get the source code from git@github.com:iotexpert/Tilt2.git  This repository has tags for each of the articles which can be accessed with "git checkout part12"  You can find the Tilt Simulator at  git@github.com:iotexpert/TiltSimulator.git.

 

Story

At this point the Tilt Data Manager has all of the knowledge of the status of Tilts.  But, we have other threads which need access to that data.  How are they going to get it in a thread safe way?  Moreover, how am I going to provide access to this data without hardcoding specific information about the threads into the Tilt Data Manager?  In this article I add functionality to the Tilt Data Manager Thread to provide Tilt status.  Then I will test the functionality by adding test code into the ntshell.  The green boxes are the functional blocks I will touch in this update.

 

Update the Public Interface to tiltDataManager.h

As I thought about what the other threads in the system would need I decided the best thing to do would be to write their public APIs.  Here it is:

  • What Color is the “Tilt” i.e. “Pink”
  • How many Tilts are in the system (so that you can loop through them)
  • What Tilts are currently active – a bitmask
  • How many events have been seen for each Tilt
  • Please give me a copy of the current data.
/////////////// Generally callable threadsafe - non blocking
char *tdm_colorString(tdm_tiltHandle_t handle);       // Return a char * to the color string for the tilt handle
int tdm_getNumTilt();                                 // Returns the number of possible tilts (probably always 8)
uint32_t tdm_getActiveTiltMask();                     // Return a bitmask of the active handles
uint32_t tdm_getNumDataSeen(tdm_tiltHandle_t handle); // Return number of data points seen
tdm_tiltData_t *tdm_getTiltData(tdm_tiltHandle_t handle);

Create the Thread Unsafe Functions

Obviously you need to do things thread safely.  However, I decided to have a few functions which couldn’t hurt anything, but are unsafe.  (Sorry Butch… comment below if you don’t agree and Ill fix it).  To make this a bit safer, I set the data as volatile to make the compiler at least write it back into memory when things are done.

typedef struct  {
    char *colorName;
    uint8_t uuid[TILT_IBEACON_HEADER_LEN];
    volatile tdm_tiltData_t *data;
    volatile int numDataPoints;
    volatile int numDataSeen;
} tilt_t;

The functions are very straight forward.

char *tdm_colorString(tdm_tiltHandle_t handle)
{
    return tiltDB[handle].colorName;
}

int tdm_getNumTilt()
{
    return NUM_TILT;
}

uint32_t tdm_getActiveTiltMask()
{
    uint32_t mask=0;
    for(int i=0;i<NUM_TILT;i++)
    {
        if(tiltDB[i].data)
            mask |= 1<<i;
    }
    return mask;
}

uint32_t tdm_getNumDataSeen(tdm_tiltHandle_t handle)
{
    return tiltDB[handle].numDataSeen;
}

Thread Safe Communication

Now, a much better answer for thread safety.  If you have multiple tasks that need access to the same data it is very tempting to just communicate between the threads with global variables (see above).  You are just asking for trouble because one task might be partially through updating data, when it is interrupted by the RTOS scheduler.  Then another threads starts reading/writing from global variables that were incompletely updated by the suspended thread.  BAD!

So, what is a person to do?

What I typically do is use an RTOS queue to communicate between the two tasks.  In the picture below you can see that the ntshell will send a message to the Tilt Data Manager that it wants to know the data from a Tilt (using the handle) and it wants the response sent back to the “response queue”.  When the Tilt Data Manager gets this command, it will build a response and push it back into the “response queue”.  With this scheme the Tilt Data Manager does not know which thread that it is talking to, just the handle of the queue.

Create the Thread Safe Function

Inside of my code, the first step in creating the thread safe communication is to add a command to the Tilt Commands, TDM_CMD_GET_DATAPOINT

typedef enum {
    TDM_CMD_ADD_DATAPOINT,
    TDM_CMD_PROCESS_DATA,
    TDM_CMD_GET_DATAPOINT,
} tdm_cmd_t;

The response to all questions about Tilts will be in the form of a “tdm_tiltData_t” structure.  Specifically one that has been malloc’d on the heap.  To support this I will create a function that:

  1. Mallocs a new structure
  2. Copys the data
  3. Fix the averaging
  4. Returns a pointer to the new data
// This function returns a malloc'd copy of the front of the most recent datapoint ... this function should only be used 
// internally because it is not thread safe.
static tdm_tiltData_t *tdm_getDataPointCopy(tdm_tiltHandle_t handle)
{
    tdm_tiltData_t *dp;
    dp = malloc(sizeof(tdm_tiltData_t));
    memcpy(dp,(tdm_tiltData_t *)tiltDB[handle].data,sizeof(tdm_tiltData_t));

    dp->gravity = dp->gravity / tiltDB[handle].numDataPoints;
    dp->temperature = dp->temperature / tiltDB[handle].numDataPoints;

    return dp;
}

Then I update the command queue to deal with the get data message.

            case TDM_CMD_GET_DATAPOINT:
                dp = tdm_getDataPointCopy(msg.tilt);
                xQueueSend(msg.msg,&dp,10);
            break;

Then I create the public function for other threads to call the send the request for the data command.  This function is called in the context of the calling thread NOT the tiltDataManager.  It will create a temporary queue for the transaction, send the command, then sit and wait for the response.

tdm_tiltData_t *tdm_getTiltData(tdm_tiltHandle_t handle)
{
    QueueHandle_t respqueue;
    tdm_tiltData_t *rsp;

    if(handle<0 || handle>=NUM_TILT || tiltDB[handle].data == 0 )
        return 0;

    respqueue = xQueueCreate(1,sizeof(tdm_tiltData_t *));
    if(respqueue == 0)
        return 0;

    tdm_cmdMsg_t msg;
    msg.msg = respqueue;
    msg.tilt = handle;
    msg.cmd =  TDM_CMD_GET_DATAPOINT;
    if(xQueueSend(tdm_cmdQueue,&msg,0) != pdTRUE)
    {
        printf("failed to send to dmQueue\n");
    }
    xQueueReceive(respqueue,(void *)&rsp,portMAX_DELAY);
    vQueueDelete(respqueue);
    return rsp;
}

Test with the ntshell

To test things whole thing out I add a new command to ntshell.  This command just try all of the public APIs

static int usrcmd_testdm(int argc, char **argv)
{
    tdm_tiltData_t *msg;

    printf("NumTilt = %d\n",tdm_getNumTilt());
    printf("Active = %X\n",(unsigned int)tdm_getActiveTiltMask());
    
    for(int i =0;i<tdm_getNumTilt();i++)
    {
        printf("Color = %s\t#Seen=%d\t",
            tdm_getColorString(i),
            (int)tdm_getNumDataSeen(i));

        if(tdm_getActiveTiltMask() & 1<<i)
        {
            msg = tdm_getTiltData(i);
            if(msg)
            {
                printf("G=%f\tT=%d",
                    msg->gravity,
                    (int)msg->temperature);
                free(msg);
            }
        }
        printf("\n");
    }
    return 0;
}

Here it is running.  Notice that I typed the command “testdm”  And that my test setup has heard two Tilts (remember I have a test broadcaster)

In the next article Ill add the TFT Display.

EW21: Lesson 0: Introduction & Resources

Summary

Hello everyone.  This is lesson 0 of a series of 7 lessons about creating an IoT application using the Infineon ModusToolbox Software Environment to create a WiFi enabled drone.  In the next two hours we will build a remote control that uses the PSoC 6 MCU, WiFi, MQTT, CapSense and a 3-D Magnetic Joystick.  Then we will build the drone which will use a PSoC 6, WiFi, MQTT, CapSense and a BLDC motor controller.

What I will do today is take you lesson by lesson through the class and talk about how it all works and what you need to do.  When I built the class it was absolutely my goal to have every button click and line of code described.  That being said,  it is likely that I made some errors.  So, during the class you will be able to send messages to my team who will answer the questions, or ask me and I’ll answer live.  If you missed the class, that’s OK, you will be able to watch it on replay.  In addition if you have a question after the live stream is over, leave a comment here and I’ll answer.

I will attempt to go slowly enough for you to follow along, but if I go to fast, don’t worry you should be able to follow along with the instructions on this website.

Every lesson will have this table in it and you will be able to click to follow along with the different lessons.

Embedded World 2021 - Infineon ModusToolbox PSoC 6 Drone 

 

Here is the overall system architecture:

The Remote Control

The remote control is built with two Infineon development boards

Kit Features
CY8CPROTO-062-4343W PSoC 6,CYW4343W WiFi Bluetooth Combo Radio,CapSense
TLE493D-W2B6 XENSIV 3-D Magnetic Sensor

Here are some pictures.

CY8CPROTO-062-4343W.  The top left of the board is a KitProg programmer.  The middle third on the left is the 4343W and the PSoC 6.  The bottom right are the CapSense buttons.  Just to the right of the programmer is a SD Card holder and S512FL Quad SPI Flash.

This the the top of the TLE-493D-W2B6 3-D magnetic sensor board.  On the far right, the tiny 6 pin chip is the actual sensor.  The big hole to the left of the sensor is to mount a magnet.  The two chips on the left are used as a bridge to USB (if you are developing).  I attach to this device using the I2C interface pins.

Here is the back where you can see the SCL, SDA, Power and Ground labeled.  Unfortunately, they are in the wrong order to plug directly into the PSoC kit so Greg had to make a little wire switcher.

Here is the board with the really really cool 3-d printed joystick.  Simply two pieces of plastic with a magnet in the bottom.

This picture show how the magnetic sense board is mounted onto the kit

Here is the whole thing assembled.

The Crazy Drone

The drone was built with

Kit Features
CY8CKIT-062S2-43012 PSoC 6 + CYW43012 Low Power Bluetooth WiFI Combo
TLE9879WXA40 BLDC Control Shield

Here is a picture of the CY8CKIT-062S2-43012.  On the far right in the middle is the PSoC 6 and CYW43012.  In the lower right are the CapSense Buttons and Slider.  The KitProg IC is just below the top Arduino Header.

In order to drive the BLDC motor I use the TLE9879WXA40 Motor Shield.  This has everything needed to do Field Oriented Control of a 3-phase BLDC motor.  The Blue, Green and White wires are the 3-phases of the BLDC.  The Red and Black are simply +12V and Ground.  You interface from the PSoC to the BLDC shield via a SPI interface (attached to the Arduino pins).

The BLDC motor is mounted into a 3-d printed holder.  The mount is attached to hollow carbon fiber tubes that run on bearings that you see below.  The wires run down through the tubes.

At the bottom, the blue box just provides +12v to the drone and the PSoC board.

Here is a closer picture of the BLDC motor.

Here is a picture of the whole crazy thing running.  If you look in the background you can see a top secret new PSoC motor controller.  Is that an Easter egg?

And yes, it will cut your fingers off if you aren’t careful.

Resources

You will need a few things for this class:

ModusToolbox Software Environment

You can download Modus Toolbox from here

CY8CPROTO-062-4343W

You can read all about this development kit on the website or in the KitGuide

CY8CKIT-062S2-43012

You can read all about this development kit on the website or in the KitGuide

TLE9879QXA40 BLDC Motor Controller

You can read all about this development kit on the website or in the KitGuide

XENSIV TLE493D

You can read all about this development kit on the website or in the KitGuide