Summary

This article discusses a detailed Bluetooth Capture using Linux and a Frontline Bluetooth Scanner.  With those tools I discover the cause of my bug.

This series is broken up into the following 12 articles with a few additional possible articles. 

Tilt Hydrometer (Part 1) Overview & Out-of-Box

Tilt Hydrometer (Part 2) Architecture

Tilt Hydrometer (Part 3) Advertising Scanner

Tilt Hydrometer (Part 4) Advertising Packet Error?

Tilt Hydrometer (Part 5) Tilt Simulator & Multi Advertising iBeacons

Tilt Hydrometer (Part 6) Tilt Simulator an Upgrade

Tilt Hydrometer (Part 7) Advertising Database

Tilt Hydrometer (Part 8) Read the Database

Tilt Hydrometer (Part 9) An LCD Display

Tilt Hydrometer (Part 10) The Display State Machine

Tilt Hydrometer (Part 11) Draw the Display Screens

Tilt Hydrometer (Part 12) CapSense

Tilt Hydrometer: LittleFS & SPI Flash (Part ?)

Tilt Hydrometer: WiFi Introducer (Part ?)

Tilt Hydrometer: WebServer (Part ?)

Tilt Hydrometer: Amazon MQTT (Part ?)

Tilt Hydrometer: Printed Circuit Board (Part ?)

You can get the source code from git@github.com:iotexpert/Tilt2.git  This repository has tags for each of the articles which can be accessed with "git checkout part12"  You can find the Tilt Simulator at  git@github.com:iotexpert/TiltSimulator.git.

 

Where is the Tilt?

As I was trying to get the program running, I was faced with a problem that sent me down a serious serious rabbit hole.  The problem was that I could see the Tilts on the iPhone App, but they were not printing out on the screen.  But why?

I ran Light Blue, but there are so many Bluetooth Devices in range that I couldn’t see a way to sort it out.

What in the world is “hcidump”

On Karl’s blog he writes that you should use hcitool and hcidump to view what is going on.  Well, I am mostly an embedded guys, but OK.

I first install Linux on a Raspberry Pi 3B+ which has a Cypress CYW43455 WiFi Bluetooth Combo device onboard.  In this configuration the Bluetooth framework is split into a controller running on the CYW43455 and a Host Stack running on the Raspberry Pi Linux chip BCM2837B0.  The Host and the Controller communicate with each other through a UART.  The protocol that is used on that UART is called “HCI”

Specifically, what Karl told you to do is:

1. In one window tell the Bluetooth Controller to start looking for advertising packets “lesscan”
2. In another windows, start snooping on the HCI bus and printing out the packets in hex

In the picture below you can see that I first run “hcitool dev” to get a list of the Bluetooth Devices.  It turns on out the Raspberry Pi configuration that I have there is only one bluetooth device and it is called “hci0”.  Then I use “hcitool -i hci0 lescan” to send a command to the controller to start BLE scanning.  The hcitool command will report just the MAC address and possibly the name (if it happens to be in the advertising packet).  Its too bad that hcitool lescan doesn’t have a switch to display the advertising packets.

In another window I run “sudo hcidump -R > out.txt”.  The hcidump command will “snoop” on the HCI uart and will print out all of the raw packets going between the host and the controller.   After a while, I open on the out.txt and start looking through the raw data to find my Tilt.  I recognize the Tilt by looking for the UUID in the iBeacon which is “A4 95 BB …”

But what is all of the other stuff on that line?  For that answer we will need to dig through the Bluetooth core spec.  If you look in Volume 4 (HCI), Part A (UART Transport Layer), Chapter 1 (General) it starts with a nice picture of the system.

Then in Chapter 2 (Protocol) it says:

So, first byte, the 0x04 means that this is an “HCI Event Packet”, meaning this is something that happened on the controller which is going back to the Host.  In fact if you look at the log above you will see that the lines are preceded by a “>” which means packets coming into the Host.  Now we know that there is an event packet, so we follow further down into the spec in Part “E” which describes the event packet.

The first byte, the “0x3E” is an event code and the 0x2A is the length of the packet (count from the 0x02 through to the end)

Keep digging further down into the spec to find out what the “event code 0x3E” and you will find that it is a “LE Meta event”.

OK so that means that the “02” is the “Subevent_Code”.  OK keep digging and you will find that the 02 Subevven_Code is “HCI_LE_Advertising_Report”.  Then it gives you a list of what data will follow.

So, the 02 is an advertising report.  The 01 says that there is only 1 advertising report.  Q: Why is this a parameter?  A: Because the controller can save a few bytes by clubbing multiple advertising reports into 1 HCI packet.  Which it did not do in this case.

Next is the 03 which is the “Event_Type[i]”.  In this case it says that this thing is non connectable undirected advertising. (more on this later)

OK what about 01 …

The 01 says that it is a random address and then the next 6 bytes is the MAC address.

Now the 0x1E is the length of the data in the advertising packet

The actual advertising pack is next.

Recall the format from section 11.

The 02 01 04 is the first field of the advertising packet and it represents

• 02 – length of the field
• 01 – «Flags»
• 04 – flags

You can find the 01 from the “Assigned numbers and GAP”

Then when you look in the Core Specification Supplement Part A section 1.3 you will find that the “4” means “BR/EDR not supported”

The next field is 0x1A in length and a field type of “0xFF” – Ah…. manufacture specific data

The 4C 00 is Apples Company Identifier Code.

Which you can find on the Bluetooth Sig Company Identifiers page.

From the previous article we know the rest is an iBeacon

• 02 – iBeacon subtype
• 0x15 – length
• Then the A4…. is UUID which maps to a “Pink” Tilt

Then we get the Temperature 0x004F (BIG ENDIAN!!!!) also known as 79 Degrees F (see Imperial… my people)

Then the gravity 0x03FD also known as 1.021 – NOTICE THIS IS BIG ENDIAN!!!!

Then transmit power C5 (also known as -59)

Finally the RSSI 0xA5 which is also known as -91 (2’s complement 1 byte)

Debugging the Linux Bluetooth

I had a few problems getting this going which led me to the door of some interesting questions (which I won’t answer here).  But, here are the problems and solutions.

Q1 When I “hcidump -R” I only get this:

A1 You need to be root to see the raw data.  Run “sudo hcidump -R”

Q2: I don’t see my device?

A2: If you start the scan before your run the hcidump you might miss the device.  When you scan try running “hcitool lescan –duplicates” which will turn off duplicate filtering.

Q3: I get “Set scan parameters failed: Input/output error”

A3: I am not totally sure I have this right because I was not able to recreate the problem (which I had in spades when I started).  But, try doing a “sudo hciconfig hci0 down” followed by a “hciconfig hci0 up”

Q4: I read that “hcitool” and “hcidump” are deprecated.

A4: I read the same thing.  If you don’t have them try “sudo apt-get install bluez-tools”.  I would like to know the “real” answer to this question

Q5: How does hcidump get the packets from the HCI?

A5: I have no idea.  But I would like to know that answer to this question

Q6: I tried “btmon”.  Does that work?

A6: Yes

Q7: What is the bluetoothd?

A7: I dont know.

Q8: Do hcidump and hcitools talk to the BluetoothD or do they talk to dbus or do they talk to the kernel through a socket?

A8: I don’t know

Q9: Why can’t I see the Tilt in my advertising scanner

A9: Keep reading 🙂

Frontline Bluetooth Scanner

I can see the Tilt in the Linux and on the iPhone,  but I still can’t see it in my AnyCloud project.  Why?  The next thing that I did was get a Frontline Bluetooth Sniffer.  When I started capturing packets I could see the device.  Here is the “Tilt”

And here is the “Tilt Repeater”

Fix the Advertising Scanner

After more digging I figured it out.  Remember from earlier that the Tilt advertises as “Non connectable”.  Well it turns out that when I built the advertising scanner I used the function “wiced_bt_ble_scan”.  This function was put into WICED to use for the connection procedure.  In other words it FILTERS devices that are non connectable.  In order to see those devices you need to call “wiced_bt_ble_observe”.  Wow that was a long article to explain that one little bug.  Oh well it was a fun learning experience.

Summary

In this article I will build the basic project and then add the scanner code to look for Tilt Hydrometers that are broadcasting the iBeacon.  It will decode the data and print out Gravity and Temperature.

This series is broken up into the following 12 articles with a few additional possible articles. 

Tilt Hydrometer (Part 1) Overview & Out-of-Box

Tilt Hydrometer (Part 2) Architecture

Tilt Hydrometer (Part 3) Advertising Scanner

Tilt Hydrometer (Part 4) Advertising Packet Error?

Tilt Hydrometer (Part 5) Tilt Simulator & Multi Advertising iBeacons

Tilt Hydrometer (Part 6) Tilt Simulator an Upgrade

Tilt Hydrometer (Part 7) Advertising Database

Tilt Hydrometer (Part 8) Read the Database

Tilt Hydrometer (Part 9) An LCD Display

Tilt Hydrometer (Part 10) The Display State Machine

Tilt Hydrometer (Part 11) Draw the Display Screens

Tilt Hydrometer (Part 12) CapSense

Tilt Hydrometer: LittleFS & SPI Flash (Part ?)

Tilt Hydrometer: WiFi Introducer (Part ?)

Tilt Hydrometer: WebServer (Part ?)

Tilt Hydrometer: Amazon MQTT (Part ?)

Tilt Hydrometer: Printed Circuit Board (Part ?)

You can get the source code from git@github.com:iotexpert/Tilt2.git  This repository has tags for each of the articles which can be accessed with "git checkout part12"  You can find the Tilt Simulator at  git@github.com:iotexpert/TiltSimulator.git.

 

Build the Basic Project

Start by making a new project using the CY8CKIT-062S2-43012.  I chose this kit because that was the one sitting on my desk at the time.  Any of the kits with the Arduino headers where I can plug in the TFT will work.

I typically start from the IoT Expert FreeRTOS NTShell Template.  Ill do that here as well.  Notice that I call the project Tilt2.  You can find this project on GitHub.  git@github.com:iotexpert/Tilt2.git .  I will also be putting in tags for each article so you can look at the code for each article.  To see the tags do “git tags”

After the project is made run “make modlibs” so that you can add some libraries.  Specifically the bluetooth-freertos and btstack libraries.

Now that I have all of the libraries I need, run “make vscode” to build the project files for vscode.

If you remember from the architecture diagram, I will have a task to manage the bluetooth world.  Inside of the btutil library I have a template to start with.  It also includes the stuff you need in main.c to start the stack.

You need to make two changes to the Makefile.  Well you dont have to change the name of the App … but I typically do.  You do need to add the FREERTOS and WICED_BLE components to your project.

APPNAME=Tilt2


....


COMPONENTS=FREERTOS WICED_BLE

Now, edit main.c to include some stuff.

#include "bluetoothManager.h"
#include "cycfg_bt_settings.h"
#include "bt_platform_cfg_settings.h"

And start the bluetooth stack.

    cybt_platform_config_init(&bt_platform_cfg_settings);
    wiced_bt_stack_init (app_bt_management_callback, &wiced_bt_cfg_settings);

You need to get the bluetooth configuration files.  To do that run the Bluetooth Configurator by running “make config_bt”.  Click the paper to make a new configuration.

Pick out the PSoC6 MCU with 43xxxx Connectivity.

Go to the GAP Settings tab and add the Observer configuration

Then set the low duty scan parameters to 60ms and turn off the timeout.

Save your configuration.  Notice that I call it tilt2.

Make sure that everything works with a build it and program.  You will have a working shell, blinking LED and the Bluetooth Task started.  Here is what the output looks like.

iBeacon Packet Format

On the Tilt Hydrometer FAQ they give you a link to Karl Urdevics blog where he describes the way that the Tilt works.  Pretty simple, each Tilt Hydrometer is hardcoded to a color.  Each “color” sends out an iBeacon with the Temperature, Gravity and Transmit Power.  iBeacon is just an Apple specified format for a BLE advertising packet.  You can read about the format of the BLE advertising packet here.  But, in summary, a BLE device can advertise up to 31 bytes of data.  That data is divided into fields of the following form:

• Field Type (1 byte) – The type is one of the Bluetooth GAP Assigned Numbers
• Length (1 byte)
• Length -1 bytes

There are a bunch of specific format types.  However, the one that we are looking for is the iBeacon format.  This format was defined by Apple and is implemented as data in the “«Manufacturer Specific Data»” which has the field code 0xFF.  The Manufacturer Specific Data field type is divided into

• 16-bit Manufacturers ID
• Data

For the iBeacon the Manufacturer ID is 0X004C – which is Apple’s ID.  They further subdivided the data.  Here is the complete format of that data.

It turns out that Tilt “hardcodes” the UUID during manufacturing for the color of the Tilt (remember each Tilt is one of 8 colors)

Red:    A495BB10C5B14B44B5121370F02D74DE
Green:  A495BB20C5B14B44B5121370F02D74DE
Black:  A495BB30C5B14B44B5121370F02D74DE
Purple: A495BB40C5B14B44B5121370F02D74DE
Orange: A495BB50C5B14B44B5121370F02D74DE
Blue:   A495BB60C5B14B44B5121370F02D74DE
Yellow: A495BB70C5B14B44B5121370F02D74DE
Pink:   A495BB80C5B14B44B5121370F02D74DE

Add Advertising Observer

Now that we have a working project template, and we know what we are looking for in the BLE advertising land, I’ll setup some data structures to map of the colors and UUIDs.

#define TILT_IBEACON_HEADER_LEN 20
#define TILT_IBEACON_DATA_LEN 5
typedef struct  {
    char *colorName;
    uint8_t uuid[TILT_IBEACON_HEADER_LEN];
} tilt_t;

// Apple Bluetooth Company Code 0x004C
// iBeacon Subtype = 0x02
// Length = 0x15
#define IBEACON_HEADER 0x4C,0x00,0x02,0x15

static tilt_t tiltDB [] =
{
    {"Red",    {IBEACON_HEADER,0xA4,0x95,0xBB,0x10,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE}},
    {"Green" , {IBEACON_HEADER,0xA4,0x95,0xBB,0x20,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE}},
    {"Black" , {IBEACON_HEADER,0xA4,0x95,0xBB,0x30,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE}},
    {"Purple", {IBEACON_HEADER,0xA4,0x95,0xBB,0x40,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE}},
    {"Orange", {IBEACON_HEADER,0xA4,0x95,0xBB,0x50,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE}},
    {"Blue"  , {IBEACON_HEADER,0xA4,0x95,0xBB,0x60,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE}},
    {"Yellow", {IBEACON_HEADER,0xA4,0x95,0xBB,0x70,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE}},
    {"Pink"  , {IBEACON_HEADER,0xA4,0x95,0xBB,0x80,0xC5,0xB1,0x4B,0x44,0xB5,0x12,0x13,0x70,0xF0,0x2D,0x74,0xDE}},
};
#define NUM_TILT (sizeof(tiltDB)/sizeof(tilt_t))

Then I will create and advertising callback that will

• Look for the manufacturer specific data field in the advertising packet
• If the length of that field is 25 then we have found a possible iBeacon
• Loop through the different possible Tilt’s
• Compare the data in the advertising packet against the data + UUID in the packet
• If it matches then decode the Gravity, txPower and Temperature and print it

static void btm_advScanResultCback(wiced_bt_ble_scan_results_t *p_scan_result, uint8_t *p_adv_data )
{
	if (p_scan_result == 0)
		return;

	uint8_t mfgFieldLen;
	uint8_t *mfgFieldData;
	mfgFieldData = wiced_bt_ble_check_advertising_data(p_adv_data,BTM_BLE_ADVERT_TYPE_MANUFACTURER,&mfgFieldLen);
    
	if(mfgFieldData && mfgFieldLen == TILT_IBEACON_HEADER_LEN + TILT_IBEACON_DATA_LEN)
    {
        for(int i=0;i<NUM_TILT;i++)
        {
            if(memcmp(mfgFieldData,tiltDB[i].uuid,TILT_IBEACON_HEADER_LEN)==0)
            {
                float gravity = ((float)((uint16_t)mfgFieldData[22] << 8 | (uint16_t)mfgFieldData[23]))/1000;
		        int temperature = mfgFieldData[20] << 8 | mfgFieldData[21];
                int8_t txPower = mfgFieldData[24];

                printf("Found Color=%s Gravity=%f Temperature = %d txPower=%d\n",tiltDB[i].colorName,gravity,temperature,txPower);
                break;
            }
        }
    }
}

In the Bluetooth Management callback I want to turn on scanning (actually observing).

        case BTM_ENABLED_EVT:
            if (WICED_BT_SUCCESS == p_event_data->enabled.status)
            {
				wiced_bt_ble_observe(WICED_TRUE, 0,btm_advScanResultCback);
            }
            else
            {
            	printf("Error enabling BTM_ENABLED_EVENT\n");
            }

To test this thing I brought a black tilt into my office.  As soon as the PSoC saw it, packets starting coming out on the screen about once per second.  The first thing to notice is that they broadcast some crazy data at the start.  That means I should be careful with the error checks (something which if you are a reader you know that I am not always perfect 🙂 .  It is also interesting to see that they broadcast 5 packets at 5dBm, then 5 at -59dBm.

Now that we have data coming out, in the next article Ill address a couple of funky things that I noticed.

Summary

This article is a walk through of the architecture of the PSoC 6 – AnyCloud firmware which I will implement for my Tilt Hydrometer IoT application

This series is broken up into the following 12 articles with a few additional possible articles. 

Tilt Hydrometer (Part 1) Overview & Out-of-Box

Tilt Hydrometer (Part 2) Architecture

Tilt Hydrometer (Part 3) Advertising Scanner

Tilt Hydrometer (Part 4) Advertising Packet Error?

Tilt Hydrometer (Part 5) Tilt Simulator & Multi Advertising iBeacons

Tilt Hydrometer (Part 6) Tilt Simulator an Upgrade

Tilt Hydrometer (Part 7) Advertising Database

Tilt Hydrometer (Part 8) Read the Database

Tilt Hydrometer (Part 9) An LCD Display

Tilt Hydrometer (Part 10) The Display State Machine

Tilt Hydrometer (Part 11) Draw the Display Screens

Tilt Hydrometer (Part 12) CapSense

Tilt Hydrometer: LittleFS & SPI Flash (Part ?)

Tilt Hydrometer: WiFi Introducer (Part ?)

Tilt Hydrometer: WebServer (Part ?)

Tilt Hydrometer: Amazon MQTT (Part ?)

Tilt Hydrometer: Printed Circuit Board (Part ?)

You can get the source code from git@github.com:iotexpert/Tilt2.git  This repository has tags for each of the articles which can be accessed with "git checkout part12"  You can find the Tilt Simulator at  git@github.com:iotexpert/TiltSimulator.git.

 

Story

I started this whole thing by just writing some code without really thinking about the architecture too much.   I had been wanting to try the Bluetooth SDK inside of AnyCloud and I was just focused on how that worked inside of the PSoC 6.  As I wrote the code, I naturally implemented with a common design pattern of

1. Individual tasks have responsibility for a specific hardware block
2. Each task has a command queue to which other tasks can send commands
3. Tasks that need to have data fed to them by other tasks have a data queue

As I worked on the code and things started to get a bit more involved I decided that I had better draw a picture of the system.  This picture served two main purposes.

1. It kept me on track
2. I knew that I was going to need it for this series of articles

Here is the architecture of what I have implemented for this series of articles.  As of this writing I have not started started not the wifi or the file system part of the implementation but I think that my design will work.

The system has 7 tasks:

There are two places in the picture above where my lines have to cross, something that I really hate.  So I decided to try a different picture to explain the system.  The picture below describes the tasks, what data they own, what hardware they own, what the inputs are and what the outputs are.

In the next article Ill start the basic project and create an advertising observer.