Summary

In this series of articles I am building a Bluetooth Low Energy Scanner using the Cypress/Infineon AnyCloud SDK running on a PSoC 6 and CYW43xxx.  In Part 8 I will turn on the ability to filter duplicate advertising packets.

Story

In the previous article I added the ability to record advertising packets.  The problem is, of course, that many devices are blasting out advertising packets, which will quickly overwhelm you.  I suppose more importantly it will overwhelm the packet buffer.  Most of the device are just advertising their presence, so they send the same data over and over.  Some devices alternate between a small number of different advertising packets, e.g. an iBeacon then and Eddystone beacon.

The way that the filter will work is that I will update the “add” function to search through all of the packets that device has launched, then if I have seen the packet before (Ill use a memcmp) then I will just keep a count of how many times I have see that packet.

The other thing that needs to happen is for me to add a “filter” command so that I can turn on packet filtering on a device by device basis.

And I need to fix the printing to use the new filtered packet database.

There are

All of the code can be found at git@github.com:iotexpert/AnyCloudBLEScanner.git and https://github.com/iotexpert/AnyCloudBLEScanner.git

There are git tags in place starting at part 5 so that you can look at just that version of the code.  "git tag" to list the tags.  And "git checkout part6" to look at the part 6 version of the code.

You can also create a new project with this is a template if you have the IoT Expert Manifest Files installed

Update the Data Structures

In order to do the filters, and keep track of the data I will add

1. A “count” field to the packet data structure
2. A “filter” boolean field to the device data structure.

typedef struct {
    uint8_t *data;
    int count;
    struct adb_adv_data_t *next;
} adb_adv_data_t;

typedef struct {
    wiced_bt_ble_scan_results_t *result;
    bool record;
    bool filter;
    int numSeen;
    int listCount;
    adb_adv_data_t *list;
} adb_adv_t ;

Update the Add Function

In the function “static void adb_db_add(wiced_bt_ble_scan_results_t *scan_result,uint8_t *data)” which is called every time I see a new advertising packet I will need to handle four different cases:

1. The first time you see a device
2. A device you have seen AND you are recording AND have the filtering turned on
3. A device you are recording but not filtering
4. A device you have seen, but are not recording or filter.

In the first case, a device you haven’t seen before, you need to

1. Automatically turn on the filtering
2. Initialize the counts to 1
3. Do the other initialization (as before)

    // If it is NOT found
    if(entry == -1)
    {
        adb_database[adb_db_count].result = scan_result;
        adb_database[adb_db_count].listCount = 1;
        adb_database[adb_db_count].record = false;
        adb_database[adb_db_count].filter = true;
        adb_database[adb_db_count].numSeen = 1;

        adb_adv_data_t *current = malloc(sizeof(adb_adv_data_t));
        current->next = 0;
        current->data = data;
        current->count = 1;

In the case where you have

1. See the device before
2. You have room in the record buffer
3. And you are in record mode

You will then decide if you are filtering.

Then you will iterate through all of the packets and compare the data to the data you just received.  If there is a match then you update the count, free the duplicate data and return.

    else if(adb_database[entry].record && adb_recording_count<ADB_RECORD_MAX && adb_recording)
    {
        adb_database[entry].numSeen += 1;

        if(adb_database[entry].filter) // if filtering is on.
        {
            int len = btutil_adv_len(data); 
            for(adb_adv_data_t *list = adb_database[entry].list;list;list = (adb_adv_data_t *)list->next)
            {
                if(memcmp(list->data,data,len) == 0) // Found the data
                {
                    list->count += 1;
                    printf("Count = %d\n",list->count);
                    free(data);
                    free(scan_result);
                    return;
                }
            }
        }

If you have not see the data before, then you need to add it to the linked list.

        adb_adv_data_t *current = malloc(sizeof(adb_adv_data_t));
        current->next = (struct adb_adv_data_t *)adb_database[entry].list;
        current->data = data;
        current->count = 1;

If you are not recording and not filtering, just increment counts.

    else
    {
        adb_database[entry].numSeen += 1;
        adb_database[entry].list->count += 1;

Add a “filter” Command

I want the ability for a user to “filter all” or “filter clear” or “filter #” – just like we did with watch.  So, add the #defines and new function to advDatabase.h

#define ADB_FILTER_ALL -1
#define ADB_FILTER_CLEAR -2
void adb_filter(int entry);

Then add the new filter command in advDatabase.c

typedef enum {
    ADB_ADD,
    ADB_PRINT_RAW,
    ADB_PRINT_DECODE,
    ADB_WATCH,
    ADB_ERASE,
    ADB_RECORD,
    ADB_FILTER,
} adb_cmd_t;

I will use the adb_queueCmd function that I created in the last article.

inline void adb_filter(int entry) { adb_queueCmd(ADB_FILTER,(void*)entry,(void *)0); }

The filter command has three cases

1. All – turn the bool for all on
2. Clear – turn the bool for all off
3. Just a specific number – toggle that specific number

static void adb_db_filter(int entry)
{
    if(entry == ADB_FILTER_ALL)
    {
        for(int i=0;i<adb_db_count;i++)
        {
            adb_database[i].filter = true;
        }
        return;
    }

    if(entry == ADB_FILTER_CLEAR)
    {
        for(int i=0;i<adb_db_count;i++)
        {
            adb_database[i].filter = false;
        }
        return;
    }

    if(entry > adb_db_count-1 || entry < ADB_WATCH_CLEAR)
    {
        printf("Record doesnt exist: %d\n",entry);
        return;      
    }
    adb_database[entry].filter = !adb_database[entry].filter; 

}

And you need to fix up the main command processor

                case ADB_FILTER:
                    adb_db_filter((int)msg.data0);
                break;

Finally add the command to the usrcmd.c

static int usrcmd_filter(int argc, char **argv)
{

    if(argc == 2 && !strcmp(argv[1],"all"))
    {
        adb_filter(ADB_FILTER_ALL); // all
        return 0;
    }


    if(argc == 2 && !strcmp(argv[1],"clear"))
    {
        adb_filter(ADB_FILTER_CLEAR);
        return 0;
    }

    if(argc == 2)
    {
        int i;
        sscanf(argv[1],"%d",&i);
        adb_filter(i);
        return 0;
    }

    return 0;
}

Fix the Printing

Now we nee to fix the printing.  I want to add an indicate of the filtering to the output.  Remember from the previous article I indicated “Watch” with a “*”.  When I looked at it, I decided that I should indicate filter with an “F” and watch with a “W”.  So I fix that.

        printf("%c%c%02d %05d %03d MAC: ",adb_database[i].record?'W':' ',
            adb_database[i].filter?'F':' ',
            i,adb_database[i].numSeen,adb_database[i].listCount);
        btutil_printBDaddress(adb_database[i].result->remote_bd_addr);
        switch(method)
        {

Then I test the “filter” command

Fix the Printing Part (2)

As I noodled on how to change the printing I decide that it would be nice to sometimes print out only one packet e.g. no history and sometimes print them all out e.g. history.  So, I add a new parameter to the function called “history”

static void adb_db_print(adb_print_method_t method,bool history,int entry)

As I looked at the printing code, I decided that it would be better to have a new function to print only one entry.  I suppose that I could have left the code inline, but I thought that intent was clearer.

static void adb_db_printEntry(adb_print_method_t method, int entry, adb_adv_data_t *adv_data)
{
    printf("%c%c%02d %05d %03d MAC: ",adb_database[entry].record?'W':' ',
    adb_database[entry].filter?'F':' ',
    entry,adb_database[entry].numSeen,adb_database[entry].listCount);
    btutil_printBDaddress(adb_database[entry].result->remote_bd_addr);

    switch(method)
    {
    
    case ADB_PRINT_METHOD_BYTES:
        printf(" Data: ");
        btutil_adv_printPacketBytes(adv_data->data);
    break;

    case ADB_PRINT_METHOD_DECODE:
        printf("\n");
        btutil_adv_printPacketDecode(adv_data->data);
    break;
    } 
    printf("\n");

}

With the new function in place I now need to update the print function to call the new entry function.  Printing the history is just a matter of iterating through the linked list.

static void adb_db_print(adb_print_method_t method,bool history,int entry)
{
    int start,end;
 
    if(entry < 0)
    {
        start = 0;
        end = adb_db_count;
    }
    else
    {
        start = entry;
        end = entry+1;
    }

    if(end>adb_db_count)
        end = adb_db_count; 

    for(int i=start;i<end;i++)
    {
        if(history) // then iterate through the linked list print all of the packets
        {
            for(adb_adv_data_t *list = adb_database[i].list;list;list = (adb_adv_data_t *)list->next)
            {
                adb_db_printEntry(method,i,list);    
            }
        }
        else // Just print the first packet in the list
            adb_db_printEntry(method,i,adb_database[i].list);
    }
}

Now, I need to update all of the calls to adb_db_print to have the new history parameter.  First, I made the decision that when you “print” from the command line that you are interested in the history.

                case ADB_PRINT_RAW:
                    adb_db_print(ADB_PRINT_METHOD_BYTES,true,(int)msg.data0);
                break;
                case ADB_PRINT_DECODE:
                    adb_db_print(ADB_PRINT_METHOD_DECODE,true,(int)msg.data0);
                break;

But when you are printing out the packet for a new device don’t print out the history

            adb_db_print(ADB_PRINT_METHOD_BYTES,false,adb_db_count-1);

Program and Test

After I program my development kit, I start by typing “watch all”.  Very quickly, at my house, you can see that a bunch of devices are discovered.  You can see that all of these have a “W” (meaning that I am watching them) and an “F” meaning they are filtering out duplicates.  Then I type “record” to turn on recording. After a minute I turn off recording then do the print you can see below.

A couple of things to notice.

1. Device #4 (which I highlighted) appears to be sending out a pattern of alternating packets.  See that I have heard 3335 packets yet there are only two in the buffer
2. You can see device 11 seems to be sending out 16 different packets.  Why?  I don’t know.

But we can “decode 11” to try to figure it out.  You can see that it is advertising manufactures specific data with Apple’s UUID which I happen to know is 0x004C.  But why?  I don’t know.

I really want to move onto a new series of articles… but there are two functions which I will add to the program.  Stay tuned for what they do.