Stupid Python Tricks: Implementing a c-switch

Summary

A discussion of a simple method in Python to implement the equivalent of a c programming style switch.

Story

Recently, I was working on an example where I would have used something like this if I was programming in C (emphasis on like)

#include <stdio.h>

typedef enum {
    op0,
    op1,
    op2,
    op3
} opcodes_t;

int main(int argc,char **argv)
{
    opcodes_t excode = op1;

    switch(excode)
    {
        case 0:
            printf("opcode 0");
            break;
        case 1:
            printf("opcode 1");
            break;
        case 2:
            printf("opcode 2");
            break;
        case 3:
            printf("opcode 3");
            break;

    }

}

And, as you know, I am not a real Python programmer.  I tried switch… obviously to no avail.  So now what?  Well in the example above I have

  • A list of keys, the enumerated opcode_t
  • That have a values e.g. “opcode 0” etc.

Sounds like a dictionary.  Here is the equivalent code in Python:

opcode_t = {
    0:"opcode 0",
    1:"opcode 1",
    2:"opcode 2",
    3:"opcode 3",
}

ex1 = 1

print(opcode_t[ex1])

But what happens if a key is missing?  The bracket[] method of a dictionary is equivalent to the dictionary method “get”.   The “get” method has a default case if they key is missing from the dictionary.  Here is the example code:

opcode_t = {
    0:"opcode 0",
    1:"opcode 1",
    2:"opcode 2",
    3:"opcode 3",
}

ex1 = 1

# will return "not found" if the key is not in the dictionary
print(opcode_t.get(ex1,"Not Found"))

What if you want to do something?  Values in dictionaries can be “function pointers” (is what I would call them in C).  They are probably properly called references in Python.  Regardless, here is some absurd code that demonstrates the example.

def opcode0():
    return "opcode 0"

def opcode1():
    return "opcode 1"

def opcode2():
    return "opcode 2"

def opcode3():
    return "opcode 3"

opcode1_t = {
    0:opcode0,
    1:opcode1,
    2:opcode2,
    3:opcode3,
}

ex1 = 1

print(opcode1_t[ex1]())

My mDNS Example

The example that lead me to this problem was decoding mDNS headers which have 8 fields of different lengths and possible values.  Here is what I actually did:

    qrText = {0:"Query",1:"Response"}
    opcodeText = {0:"Query",1:"IQuery",2:"Status",3:"reserved",4:"Notify",5:"Update"}
    aaText = {0:"Non Authoratative",1:"Authoratative"}
    tcText = {0:"No Truncation",1:"Truncation"}
    rdText = {0:"No Recursion",1:"Recursion"}
    raText = {0:"No Recursion Available",1:"Recursion Available"}
    zText = {0:"Reserved"}
    rcodeText = {
        0:"No Error",
        1:"Format Error",
        2:"Server Failure",
        3:"Name Error",
        4:"Not implemented",
        5:"Refused - probably a policy reason",
        6:"A name exists when it should not",
        7:"a resource record set exists that should not",
        8:"NX RR Set - A resource record set that should exist does not",
        9:"Not Authorized",
        10:"Not Zone",
    }

    def printHeader(self):
        print(f"id = {self.id}")
        print(f"qr = {self.qr} {self.qrText.get(self.qr,'Unknown')}")
        print(f"opcode = {self.opcode} {self.opcodeText.get(self.opcode,'Unknown')}")
        print(f"aa = {self.aa} {self.aaText.get(self.aa,'Unknown')}")
        print(f"tc = {self.tc} {self.tcText.get(self.tc,'Unknown')}")
        print(f"rd = {self.rd} {self.rdText.get(self.rd,'Unknown')}")
        print(f"ra = {self.ra} {self.raText.get(self.ra,'Unknown')}")
        print(f"z = {self.z} {self.zText.get(self.z,'Unknown')}")
        print(f"response code = {self.rcode} {self.rcodeText.get(self.rcode,'Unknown')}")
        print(f"rc question count = {self.qdcount}")
        print(f"answer count = {self.ancount}")
        print(f"name server count = {self.nscount}")
        print(f"additional record count = {self.arcount}")

PSoC 6 SDK OneWire Bus (Part 5): Round out the OWB driver

Summary

This article shows the completion of the PSoC 6 SDK One Wire Bus library.  It shows the test apparatus for evaluating DS18B20 sensors.

Story

If you remember from the previous articles, I created the one wire bus library by looking at the function calls in the DS18B20 library and reverse engineering the function prototypes.  And, as I said in the earlier article, it would have been way better to just look at the David Antliff “OWB” library.  But that is not how it went down.  I wonder how much time in the world is wasted by programmer re-implementing code that already exists?

After the first four articles, I had three functions which the DS18B20 library defined, but I had not yet implemented.

owb_ret_t owb_write_rom_code(OneWireBus *bus, OneWireBus_ROMCode romcode);

owb_ret_t owb_crc8_bytes(uint32_t val, uint8_t *buffer, uint32_t length);

void   owb_set_strong_pullup( OneWireBus *bus, bool val);

I had not implemented them because I was not totally sure how they worked.  So, I decided to go back to GitHub and see what David had done originally.  When I got to the GitHub site, https://github.com/DavidAntliff/esp32-owb/blob/master/include/owb.h, there were actually quite a few functions in his owb library that I had not implemented.

Here is the list:

owb_ret_t owb_use_crc(OneWireBus * bus, bool use_crc);
owb_ret_t owb_use_parasitic_power(OneWireBus * bus, bool use_parasitic_power);
owb_ret_t owb_use_strong_pullup_gpio(OneWireBus * bus, cyhal_gpio_t gpio);
owb_ret_t owb_read_rom(OneWireBus * bus, OneWireBus_ROMCode * rom_code);
owb_ret_t owb_verify_rom(OneWireBus * bus, OneWireBus_ROMCode *rom_code, bool * is_present);
owb_ret_t owb_write_rom_code(OneWireBus * bus, OneWireBus_ROMCode *rom_code);
uint8_t owb_crc8_byte(uint8_t crc, uint8_t data);
uint8_t owb_crc8_bytes(uint8_t crc, const uint8_t * data, size_t len);
owb_ret_t owb_search_first(OneWireBus * bus, OneWireBus_SearchState * state, bool *found_device);
owb_ret_t owb_search_next(OneWireBus * bus, OneWireBus_SearchState * state, bool *found_device);
char * owb_string_from_rom_code(OneWireBus_ROMCode *rom_code, char * buffer, size_t len);
owb_ret_t owb_set_strong_pullup(OneWireBus * bus, bool enable);

In this article I will create and/or copy the missing functions.  As I look through his implementation I also notice that we have some style differences that I will discuss.  In general I will say that his implementation is very good and any place that I did something different was just a matter of programming taste.

I was originally planning on this article taking you linearly through how I did the changes, but in reality I jumped around while I was doing the changes, that approach won’t work.  Here is the list of what I did:

  1. Add Doxygen Function Headers
  2. Move Logging Function to Library
  3. Change Commands to Enumerated Type
  4. Change Const Bus Pointer
  5. Pack the Structures
  6. Pass Structures as Pointers
  7. Driver Functions
  8. Test the Search
  9. Test the Parastitic Power

Doxygen Headers

I like using documentation that has been generated with Doxygen.  Specifically I like that the documentation is written “at the point of attack”.  But I have never actually used or generated it using Doxygen.  To make the documentation you need to put comments into your c-header files in the right format.  Here is an example from owb.h

/**
 * @brief Represents a set of the legal one wire bus commands
 */
typedef enum {
    OWB_ROM_SEARCH        =0xF0,  ///< Perform Search ROM cycle to identify devices on the bus
    OWB_ROM_READ          =0x33,  ///< Read device ROM (single device on bus only)
    OWB_ROM_MATCH         =0x55,  ///< Address a specific device on the bus by ROM
    OWB_ROM_SKIP          =0xCC,  ///< Address all devices on the bus simultaneously
    OWB_ROM_SEARCH_ALARM  =0xEC,  ///< Address all devices on the bus with a set alarm flag
} owb_commands_t;

In this case, David had done most of the work already and all I needed to do was copy/modify his headers in the few places where we had differences in the public interface to the library.  After that, I ran doxygen.  When I first did this I got an absolute boatload of warnings about places where I had not documented with comments.  If Hassane is reading he will say … “Really Alan didn’t write comments, imagine that”

arh (master *) p6sdk-onewire $ doxygen
Doxygen version used: 1.8.18
Searching for include files...
Searching for example files...
Searching for images...
Searching for dot files...
Searching for msc files...
Searching for dia files...
Searching for files to exclude
Searching INPUT for files to process...
Searching for files in directory /Users/arh/proj/owb-ds18b20/DS18B20Test/p6sdk-onewire
Reading and parsing tag files
Parsing files
Reading /Users/arh/proj/owb-ds18b20/DS18B20Test/p6sdk-onewire/README.md...
Preprocessing /Users/arh/proj/owb-ds18b20/DS18B20Test/p6sdk-onewire/owb.c...
Parsing file /Users/arh/proj/owb-ds18b20/DS18B20Test/p6sdk-onewire/owb.c...
Preprocessing /Users/arh/proj/owb-ds18b20/DS18B20Test/p6sdk-onewire/owb.h...
Parsing file /Users/arh/proj/owb-ds18b20/DS18B20Test/p6sdk-onewire/owb.h...
Building group list...
Building directory list...
Building namespace list...
Building file list...
Building class list...
Computing nesting relations for classes...
Associating documentation with classes...
Building example list...
Searching for enumerations...
Searching for documented typedefs...
Searching for members imported via using declarations...
Searching for included using directives...
Searching for documented variables...
Building interface member list...
Building member list...
Searching for friends...
Searching for documented defines...
Computing class inheritance relations...
Computing class usage relations...
Flushing cached template relations that have become invalid...
Computing class relations...
Add enum values to enums...
Searching for member function documentation...
Creating members for template instances...
Building page list...
Search for main page...
Computing page relations...
Determining the scope of groups...
Sorting lists...
Determining which enums are documented
Computing member relations...
Building full member lists recursively...
Adding members to member groups.
Computing member references...
Inheriting documentation...
Generating disk names...
Adding source references...
Adding xrefitems...
Sorting member lists...
Setting anonymous enum type...
Computing dependencies between directories...
Generating citations page...
Counting members...
Counting data structures...
Resolving user defined references...
Finding anchors and sections in the documentation...
Transferring function references...
Combining using relations...
Adding members to index pages...
Correcting members for VHDL...
Generating style sheet...
Generating search indices...
Generating example documentation...
Generating file sources...
Generating code for file owb.h...
Generating file documentation...
Generating docs for file owb.h...
Generating page documentation...
Generating docs for page md_README...
Generating group documentation...
Generating class documentation...
Generating docs for compound OneWireBus...
Generating docs for compound OneWireBus_ROMCode...
Generating docs for nested compound OneWireBus_ROMCode::fields...
Generating docs for compound OneWireBus_SearchState...
Generating namespace index...
Generating graph info page...
Generating directory documentation...
Generating index page...
Generating page index...
Generating module index...
Generating namespace index...
Generating namespace member index...
Generating annotated compound index...
Generating alphabetical compound index...
Generating hierarchical class index...
Generating member index...
Generating file index...
Generating file member index...
Generating example index...
finalizing index lists...
writing tag file...
Running plantuml with JAVA...
lookup cache used 45/65536 hits=385 misses=48
finished...
arh (master *) p6sdk-onewire $

Now I have some documentation

I will say that I wish I had a few days to really learn Doxygen as there are many many many options which I have no idea what they do.  Oh well.

Fix the Logging Function

David built the library on top of the ESP32 libraries.  This included calls to the ESP32 logging library “log.h/.c“.  All through his library he calls “ESP_LOG” like this.

ESP_LOGE(TAG, "bus is NULL");

When I first looked at this I decided to just do this to make the error messages go away.  This is just a trick that will use the c-preprocessor to replace the function call to “ESP32_LOGE” with NOTHING

#define ESP_LOGE(...)

After I sorted everything else out I went back to decide what to do.  My choices were

  1. Clone the Espressif library and “fix it”
  2. Implement the functions that David used in the OWB library
  3. Find another library

I decided to use option 3 – a logging library that I found on GitHub called “log.c” (which is a very unfortunate name when you clone it).  It is really simple and works well.  Since the Tour de France is going as I write this article I will say “chappeau rxi”.  This library is written in C99 (just normal C) and has functions which can easily replace the ESP_LOG functions.  I add it to my project with “git clone git@github.com:rxi/log.c.git”

log_trace(const char *fmt, ...);
log_debug(const char *fmt, ...);
log_info(const char *fmt, ...);
log_warn(const char *fmt, ...);
log_error(const char *fmt, ...);
log_fatal(const char *fmt, ...);

This means that I just replace “ESP_LOGE” with “log_error”.  In reality rxi did something very nice by using a feature of the compiler to insert the file/line numbers.

#define log_trace(...) log_log(LOG_TRACE, __FILE__, __LINE__, __VA_ARGS__)

This only left me with the function

ESP_LOG_BUFFER_HEX_LEVEL(TAG, scratchpad, count, ESP_LOG_DEBUG);

Which I decided to do something cheap to solve.  The original code was:

              //ESP_LOG_BUFFER_HEX_LEVEL(TAG, &scratchpad->trigger_high, 3, ESP_LOG_DEBUG);

And I replaced the two uses with:

log_debug( "scratchpad write 3 bytes:%02X%02X%02X",scratchpad[0],scratchpad[1],scratchpad[2]);

and

                //ESP_LOG_BUFFER_HEX_LEVEL(TAG, scratchpad, count, ESP_LOG_DEBUG);
                log_debug( "%02X%02X%02X%02X%02X%02X%02X%02X%02X",
                    scratchpad[0],scratchpad[1],scratchpad[2],
                    scratchpad[3],scratchpad[4],scratchpad[5],
                    scratchpad[6],scratchpad[7],scratchpad[8]);

I know it isn’t beautiful what I did, but it works.

Enumerated Command Type

When I examined the original source code, the one-wire commands were defined using #defines.

// ROM commands
#define OWB_ROM_SEARCH        0xF0  ///< Perform Search ROM cycle to identify devices on the bus
#define OWB_ROM_READ          0x33  ///< Read device ROM (single device on bus only)
#define OWB_ROM_MATCH         0x55  ///< Address a specific device on the bus by ROM
#define OWB_ROM_SKIP          0xCC  ///< Address all devices on the bus simultaneously
#define OWB_ROM_SEARCH_ALARM  0xEC  ///< Address all devices on the bus with a set alarm flag

When I did the implementation originally I chose to make the #defines into an enumerated list.  I suppose that it doesn’t really matter.  But, by enumerating the values it lets the compiler help you in situations like a switch or a function call.

typedef enum {
    OWB_ROM_SEARCH        =0xF0,  ///< Perform Search ROM cycle to identify devices on the bus
    OWB_ROM_READ          =0x33,  ///< Read device ROM (single device on bus only)
    OWB_ROM_MATCH         =0x55,  ///< Address a specific device on the bus by ROM
    OWB_ROM_SKIP          =0xCC,  ///< Address all devices on the bus simultaneously
    OWB_ROM_SEARCH_ALARM  =0xEC,  ///< Address all devices on the bus with a set alarm flag
} owb_commands_t;

Const Bus pointer

Through out the original one wire bus library, the bus pointer is defined as const.  Like this:

owb_status owb_read_rom(const OneWireBus * bus, OneWireBus_ROMCode * rom_code)

But, I wanted to use the OneWireBus structure to also store some context.  By context I mean variables (which can change) but hold state for the bus.  This included the semaphore that I used to fix the delay functions.

/**
 * @brief Structure containing 1-Wire bus information relevant to a single instance.
 */
typedef struct  
{
    cyhal_gpio_t  pin;                   ///<Pin that the bus is attached to
    cyhal_gpio_t strong_pullup_gpio;     ///<Pin that the pullup gpio is attached to
    bool use_parasitic_power;            ///<Driver is using parastic power mode
    bool use_crc;                        ///<Enable the use of crc checks on the ROM
    // Internal use only
    bool is_init;                        ///<Private
    bool detect;                         ///<Private
    SemaphoreHandle_t signalSemaphore;   ///<Private
    cyhal_timer_t bitTimer;              ///<Private
    SemaphoreHandle_t owb_num_active;    ///<Private
    uint8_t scratchBitValue;             ///<Private
} OneWireBus;

ROM Code Structure Packed

In the original library the ROMCode is a union that allows access to individual bytes, or the actual data.

typedef union
{
    /// Provides access via field names
    struct fields
    {
        uint8_t family[1];         ///< family identifier (1 byte, LSB - read/write first)
        uint8_t serial_number[6];  ///< serial number (6 bytes)
        uint8_t crc[1];            ///< CRC check byte (1 byte, MSB - read/write last)
    } __PACKED fields;             ///< Provides access via field names

    uint8_t bytes[8];              ///< Provides raw byte access

} OneWireBus_ROMCode;

The problem is there is no guarantee that the compiler will pack the family, serial number and crc.  You should tell it to with the __PACKED macro.  I would say that I think that David got luck that there was not a bug.

/**
 * @brief Represents a 1-Wire ROM Code. This is a sequence of eight bytes, where
 *        the first byte is the family number, then the following 6 bytes form the
 *        serial number. The final byte is the CRC8 check byte.
 */
typedef union
{
    /// Provides access via field names
    struct fields
    {
        uint8_t family[1];         ///< family identifier (1 byte, LSB - read/write first)
        uint8_t serial_number[6];  ///< serial number (6 bytes)
        uint8_t crc[1];            ///< CRC check byte (1 byte, MSB - read/write last)
    } __PACKED fields;             ///< Provides access via field names

    uint8_t bytes[8];              ///< Provides raw byte access

} OneWireBus_ROMCode;

C-Programming: Passing Structures as Function Arguments

In the original library David passed the rom_code structure as an argument on the stack.  But, because I started programming in the days before it was legal to pass a structure as a function argument when I did the implementation I passed a pointer.

owb_status owb_verify_rom(const OneWireBus * bus, OneWireBus_ROMCode rom_code, bool * is_present);

I wrote

owb_ret_t owb_verify_rom(OneWireBus * bus, OneWireBus_ROMCode *rom_code, bool * is_present);

Which meant that he could write

        OneWireBus_SearchState state = {
            .rom_code = rom_code,
            .last_discrepancy = 64,
            .last_device_flag = false,
        };

but I had to write

        OneWireBus_SearchState state = {
            //.rom_code = rom_code,
            .last_discrepancy = 64,
            .last_device_flag = false,
        };

        memcpy(&state.rom_code,rom_code,sizeof(OneWireBus_ROMCode));

In this case it doesn’t really matter as the structure is small.

Driver Functions

If you look at the original implementation the author has a “driver”

typedef struct
{
    const struct _OneWireBus_Timing * timing;   ///< Pointer to timing information
    bool use_crc;                               ///< True if CRC checks are to be used when retrieving information from a device on the bus
    bool use_parasitic_power;                   ///< True if parasitic-powered devices are expected on the bus
    gpio_num_t strong_pullup_gpio;              ///< Set if an external strong pull-up circuit is required
    const struct owb_driver * driver;           ///< Pointer to hardware driver instance
} OneWireBus;

Which is a structure with function pointers to talk to bus.

/** NOTE: Driver assumes that (*init) was called prior to any other methods */
struct owb_driver
{
    /** Driver identification **/
    const char* name;

    /** Pointer to driver uninitialization function **/
    owb_status (*uninitialize)(const OneWireBus * bus);

    /** Pointer to driver reset functio **/
    owb_status (*reset)(const OneWireBus * bus, bool *is_present);

    /** NOTE: The data is shifted out of the low bits, eg. it is written in the order of lsb to msb */
    owb_status (*write_bits)(const OneWireBus *bus, uint8_t out, int number_of_bits_to_write);

    /** NOTE: Data is read into the high bits, eg. each bit read is shifted down before the next bit is read */
    owb_status (*read_bits)(const OneWireBus *bus, uint8_t *in, int number_of_bits_to_read);
};

Which meant that he did this:

bus->driver->read_bits(bus, &id_bit, 1);
bus->driver->read_bits(bus, &cmp_id_bit, 1);

In my implementation I wrote functions for those things.

owb_read_bit(bus,&id_bit);
owb_read_bit(bus,&cmp_id_bit);

I don’t really think that it helped abstract the hardware because he also did this.

            gpio_set_level(bus->strong_pullup_gpio, enable ? 1 : 0);

Perhaps a driver with function pointers would have made the original port easier if I had started there?  But if so, it would have required more adherence to the original  architecture.

Test Search

A nice thing that came with his library was an implementation of the search feature which allows multiple devices to be attached to the bus.  To test this I added two sensors.

Then made a command in my console to run the test.

static int usrcmd_search(int argc, char **argv)
{
    printf("Search\n");
    OneWireBus_SearchState state;
    bool found_device;

    owb_search_first(&bus, &state, &found_device);

    do {
        if(found_device)
        {
            printf("Found Device = ");
            for(int i=0;i<8;i++)
            {
                printf("%02X ",state.rom_code.bytes[i]);
            }
            printf("\n");
        }

        owb_search_next(&bus, &state, &found_device);

    } while(found_device);

    printf("Search done\n");
    return 0;
}

Which worked perfectly.

Parasitic Power

I would like to test the functionality of the parasitic power.  Here is a schematic from the data sheet of how it work.  But I don’t have that transistor so that will be left for another day.

What is Next?

There are several things that I should do.

  1. Fix up the libraries to use the manifest files so they are available all of the time
  2. Fix up the libraries to use the dependency scheme
  3. Test the parasitic power
  4. Test the actual DS18B20 library (the original point of this whole thing)

I have found myself in the middle of a bunch of other problems, so these things will need to happen another day.

Stupid Python Tricks: C-Structures using the ctypes Module (part 2)

Summary

A discussion of overriding __new__ and __init__ to simplify the creation Python cstruct.Structure objects.

Creating a new cytpes.BigEndianStructure

Here is a simple example of a 2-byte structure using the ctypes.BigEndianStructure class.

In this example I:

  • Derive a new class called MyStruct from the BigEndianStructure
  • Declare three fields called first (4-bits), second (4-bits) and third (8-bits).
  • Create a new object of MyStruct type called “a”
  • Set the values of the three fields
  • Print it out.
import ctypes

class MyStruct(ctypes.BigEndianStructure):
    _pack_ = 1
    _fields_ = [    ("first",ctypes.c_uint8,4),
                    ("second",ctypes.c_uint8,4),
                    ("third",ctypes.c_uint8,8),
                 ]

# Create a blank MyStruct
a = MyStruct()
a.first  = 0xa
a.second = 0xb
a.third  = 0xcd
print(bytes(a))

When I run this you can see that indeed I get 0xABCD as the result.  Several comments about this:

  • Notice that it is BigEndian (which is good since that is what I declared)
  • 0xA = 4-bits, 0-xB=4-bit so 0xAB is the first byte.
(venv) $ python ex-struct.py 
b'\xab\xcd'

You can also create a new structure by calling the class method “from_buffer_copy” with parameter of bytes type.

# Create a MyStruct initialized to Hex ABCD
b = MyStruct.from_buffer_copy(b"\xab\xcd")
print(bytes(b))

When you run this, you get the same result.

(venv) $ python ex-struct.py 
b'\xab\xcd'

What I was really hoping to be able to do is create a new structure from an array of bytes like this:

# Create a new MyStruct from an Array of bytes ... this is gonna crash
c = MyStruct(b"\x0abb\xcd")
print(bytes(c))

But that crashes.

(venv) $ python ex-struct.py 
b'\xab\xcd\x00\x00'
b'\nbb\xcd'
Traceback (most recent call last):
  File "ex-struct.py", line 24, in <module>
    c = MyStruct(b"\x0abb\xcd")
TypeError: an integer is required (got type bytes)
(venv) $

And for some reason this took me a really long time to figure out.  You probably say to yourself, “Im not surprised it took him so long to figure out.  He is programming in Python so he probably isn’t very smart anyway”

Overriding __new__ & __init__

While working to understand, I ran into the article “A better way to work with raw data types in Python” which I found interesting.  Here is a screen shot of a bit of the code.

OK.  So lets add the dunder init and dunder new methods to my class.

class MyStruct1(ctypes.BigEndianStructure):
    _pack_ = 1
    _fields_ = [    ("first",ctypes.c_uint8,4),
                    ("second",ctypes.c_uint8,4),
                    ("third",ctypes.c_uint8,8),
                 ]

    def __new__(self,sb=None):
        if(sb):
            return self.from_buffer_copy(sb)
        else:
            return ctypes.BigEndianStructure.__new__(self)

    def __init__(self,sb=None):
        pass

print("Next case")

c = MyStruct1()
c.first  = 0xa
c.second = 0xb
c.third  = 0xcd
print(bytes(c))

d = MyStruct1(b'\xab\xcd')
print(bytes(d))

Now when I run it, things are good.

(venv) $ python ex-struct.py 

Next case
b'\xab\xcd'
b'\xab\xcd'

Why do I need the __init__?

So, why do I need the dunder init that doesn’t actually do anything? Presumably if I was a real python programmer I would have already known the answer.  But I’m not, so I didn’t.

The first question I had is what does the Python keyword “pass” do?  The answer is nothing.  It is a nop or void if you prefer and is there just to make the program legal as a function has to have some function.

Then onto the Python documentation where I found that if you have an __new__ method that returns an object of the subtype Python will automatically call the __init__ function.

A Conundrum

The other interesting function that the author added was added was:

def __str__(self):
  return buffer(self)[:]

This function actually crashes because there is no member called “buffer”.  I am not sure if the cause is:

  • The buffer attribute was left out of the class by the author
  • The buffer was formerly an attribute of the Python 2.x ctypes base class (this is what I suspect)

Stupid Python Tricks: C-Structures using the ctypes Module

Summary

A discussion of reading data out of stream of bytes (encoded in a C-like structure) using the Python ctypes module.  The data in the stream is a UDP packet that represents an mDNS query or request.  The purpose of this article is to explain a process for decoding bytes streams in Python

Story

While I was working on A-Class Linux implementations I fell down the rabbit hole of mDNS.  mDNS is a part of the set of protocols that make up “Zero Configuration Networking”.  In order to understand the protocol I decided to implement (partially) an mDNS server.  You can read about that protocol and my implementation – when I get done :-).  However, all of that isn’t really important to this article, but it did bring me to dig into techniques for examining bytes in Python.

I doubt that this article is canonical, but I hope that it is at least useful.  I did find quite a few partial discussions of this topic, but I had to dig into to really understand.

Python Comment
bytes A built in object to represent an immutable sequence of single bytes.
bytearray A built in object to represent a mutable sequence of single bytes.
struct A module to encode and decode bytes from c-like structures (unfortunately the byte is an atomic unit of the struct module)
ctypes A module to interface to C functions and data.  It contains a bunch of classes which can be used to interface with C-Structures (like the struct module)

There are bunches of web hits on this topic.  However, here are a few which I found useful.

Link Comment
link A basic discussion of the ctypes module and the basic classes
link A discussion of the ctypes.sizeof function
link A discussion of the bytearray
link A Better Way to Work with Raw Data Types in Python

The UDP Header for a mDNS Packet

The IETF RFC 6895 documents the header format for mDNS (and DNS) packets.  The header contains data in Big Endian format encoded into 12 bytes that are broken up into bits, several-bits, and a few 16-bit integers.  Here is snapshot from the RFC.

 

A Red Herring

OK, I admit it.  I am a C-Programmer from way back.  My first inclination to decode the bytes looked like this:

  • Using shifts and or’s to assemble the bytes into big endian uint16s e.g. line 2
  • Using bit masks and logic “and” with or’s and shifts to pick out bit fields e.g. line 12
  • Using a tower of if/elif/elif/else to decode the individual values e.g. lines

Here is my first crack at this.

    id = message[0] << 8 | message[1]
    print(f"Id = {id}")
   
    QRFlag = (message[2] & 0x80) >> 7
    if QRFlag == 0:
        QRFlagText = "QUERY"
    else:
        QRFlagText = "RESPONSE"
        
    print(f"Query Flag = {QRFlagText}")

    opCode = (0b01111000 & message[2]) >> 3
    if opCode == 0:
        opCodeText = "Query"
    elif opCode == 1:
        opCodeText = "IQUERY"
    elif opCode == 2:
        opCodeText = "Status"
    elif opCode == 3:
        opCodeText = "Reserved"
    elif opCode == 4:
        opCodeText = "Notify"
    elif opCode == 5:
        opCodeText = "Update"
    else :
        opCodeText = "Unknown"
   
    print(f"Opcode ={opCode} {opCodeText}")

    AAFlag = (message[2] & 0b00000100) >> 2
    AAFlagText = "Authoritative" if AAFlag == 1 else "Non-Authoritative"
    print(f"AA Flag = {AAFlag} {AAFlagText}")

    TCFlag = (message[2] & 0b00000010) >> 1
    TCFlagText = "Truncation" if TCFlag == 1 else "No Truncation"
    print(f"TCFlag = {TCFlag} {TCFlagText}")

    RDFlag = message[2] & 0b00000001
    RDFlagText = "Recursion" if RDFlag == 1 else "No Recursion"
    print(f"RDFlag = {RDFlag} {RDFlagText}")

    RAFlag = (message[3] & 0b10000000) >> 7
    RAFlagText = "Recursion Available" if RDFlag == 1 else "No Recursion Available"
    print(f"RAFlag = {RAFlag} {RAFlagText}")

    ZFlag =  (message[3] & 0b01110000) >> 4
    print(f"Reserved ZFlag = {ZFlag}")

    RCCode = message[3] & 0b00001111
    if RCCode == 0:
        RCCodeText = "No Error"
    elif RCCode == 1:
        RCCodeText == "Format Error"
    elif RCCode == 2:
        RCCodeText == "Server Failure"
    elif RCCode == 3:
        RCCodeText == "Name Error"
    elif RCCode == 4:
        RCCodeText == "Not Implemented"
    elif RCCode == 5:
        RCCodeText == "Refused"
    elif RCCode == 6:
        RCCodeText == "Yx Domain"
    elif RCCode == 7:
        RCCodeText == "YX RR Set"
    elif RCCode == 8:
        RCCodeText == "NX RR Set"
    elif RCCode == 9:
        RCCodeText == "Not Authorized"
    elif RCCode == 10:
        RCCodeText == "Not Zone"

    print(f"RCCode = {ZFlag} {RCCodeText}")


    QDCount = message[4]<<8  | message[5]
    ANCount = message[6]<<8  | message[7]
    NSCount = message[8]<<8  | message[9]
    ARCount = message[10]<<8 | message[11]
    print(f"Questions = {QDCount} Answers = {ANCount} Name Servers = {NSCount} Additional Records = {ARCount}")

Encoding a c-structure with Bits

I didn’t really like the above implementation.  So I kept digging.  After a while I found the ctypes module.  This lets you

  • Derive a new class from the BigEndianStructure class (line 1)
  • Pack all of the bits and bytes next to each other (line 2)
  • Specify the field names, type and optionally the length in bits (line
class dnsHeader(ctypes.BigEndianStructure):
    _pack_ = 1
    _fields_ = [    ("id",ctypes.c_uint,16),
                    ("qr",ctypes.c_uint,1),
                    ("opcode",ctypes.c_uint,4),
                    ("aa",ctypes.c_uint,1),
                    ("tc",ctypes.c_uint,1),
                    ("rd",ctypes.c_uint,1),
                    ("ra",ctypes.c_uint,1),
                    ("z",ctypes.c_uint,3),
                    ("rcode",ctypes.c_uint,4),
                    ("qdcount",ctypes.c_uint16),
                    ("ancount",ctypes.c_uint16),
                    ("nscount",ctypes.c_uint16),
                    ("arcount",ctypes.c_uint16),

    ]

When you receive data from a socket you will get a tuple that contains

  1. a “bytes” type object containing the raw bytes of the message
  2. a “tuple” containing the IP address (not relevant to this discussion)
    (message, address) = UDPServerSocket.recvfrom(bufferSize)

Now that you have the bytes you can create an object of dnsHeader type to interpret the bytes.  The ctypes class method “from_buffer_copy” will take an array of bytes that is at least the length of the structure and return an object of the type of “dnsHeader”.

    dnsh = dnsHeader.from_buffer_copy(message)

Then you can look at the individual fields like this:

print(f"id = {self.id}")

PSoC 6 Pins & the SPI Port

Summary

Recently, I have been helping a reader sort out some code that makes strings of WS2812 LEDs work.  Specifically, this code takes data from a frame buffer inside of the PSoC 6 and drives it out a SPI port via the MOSI pin. I have written about this a couple of times, but,  the new wrinkle in our code is that it allows you to use any combination of SPI port/pins on the chip.  Instead of using the configurators to setup the SPI to GPIO connection, I setup the connection using PDL to talk directly to the PSoC 6 configuration the registers.

Perhaps it is obvious to everyone how a connection from a peripheral to a GPIO works, but I thought that I would write about it anyway.  In this article I am going to show you a bunch of the documentation for PSoC as well as the PDL source code which implements the documentation.  Specifically, I will show you the PSoC 6

  1. Architecture TRM
  2. Register TRM
  3. Datasheet
  4. PDL

Architecture TRM

In the picture below, which I copied from the PSoC 6 Architecture TRM, you can see how an individual GPIO works.  Starting  at the Pin of the chip you can see that there are three connections from/to the pin (look at the green box).

  • A set of switches to/from the Analog Mux Bus which enable CapSense or Analog peripherals to talk to the Pin
  • A connection from the pin to the Analog peripherals (some Analog peripherals can attach directly to a pin and not though the Analog Mux Bus
  • A connection to the High Speed I/O Matrix (HSIOM) – for the digital peripherals

Notice that all of the signals coming into the green box from the top are DIGITAL.  All of the signals coming into the box from the bottom are Analog and the line coming into the middle of the box controls the behavior of the I/O.

For my case the SPI is one of the “Fixed Function Digital Peripherals”.  In order to get it to connect to the pin I will beed to pick out the right signal in the multiplexer that is in the HSIOM Matrix box.

When you scroll down a little bit further in the Architecture TRM, the next diagram is a more detailed description of the GPIO.  Notice that there are a bunch of configuration register bits which setup different parts of the I/O like slew rate, interrupts, drive mode etc.  Notice that the multiplexer that is connected to “out”
and “out_en” has a bunch of different possible signals.  Including “GPIO_PRTx_OUT[OUTy]” which is a register bit which is can be used for “digital write”.  For instance GPIO_PRT0_OUT[2] would be P0_2.  The other interesting thing going on here is you can see that there are really three classes of signals attached to the mutiplexer

  • The digital output pin
  • Active signals – which work while the chip is not in deep sleep
  • Deep Sleep signals – which work while the chip is in deep sleep.

On the output side you can see the two pullup and pulldown resistors, as well as the two transistors which pullup and pull down.  All of these can be configured to be connected… or not.

And finally at the bottom of the I/O you can see the analog signals.

 

If you look a little bit further down in the Architecture TRM you will find this table which describes how each of the actual pins on the multiplexer work.

PSoC 6 Register TRM

If you want to start to make specific configurations for specific pins you will need to look into the PSoC 6 Register TRM.  In that document you will find that the

Register TRM

Register TRM

PSoC 6 Datasheet

But what are all of the active and deep sleep signals?  Well if you look in the PSoC 6 data sheet you can find all of those connections.  For instance on P0.1 active signal 8 is the SCB 0 SPI Select signal 2.

PSoC 6 PDL

But really all of these register reads and writes are not really that fun.  So, Cypress provides other, less painful ways of getting things going.  Specifically,

  • void Cy_GPIO_SetHSIOM(GPIO_PRT_Type* base, uint32_t pinNum, en_hsiom_sel_t value)

or

  • Cy_GPIO_Pin_Init(GPIO_PRT_Type *base, uint32_t pinNum, const cy_stc_gpio_pin_config_t *config)

When you call both of these function you need to provide the value for the multipler either directly in the Cy_GPIO_SetHSIOM or indirectly in the Cy_GPIO_Pin_Init case where you provide it as a member of the cy_stc_gpio_pin_config_t *config structure called “hsiom”

Depending on which package you have selected you will have a file like gpio_psoc6_01_124_bga.h which will have both the generic definitions for the HSIOM multiplexer select (like this)

/* HSIOM Connections */
typedef enum
{
    /* Generic HSIOM connections */
    HSIOM_SEL_GPIO                  =  0,       /* GPIO controls 'out' */
    HSIOM_SEL_GPIO_DSI              =  1,       /* GPIO controls 'out', DSI controls 'output enable' */
    HSIOM_SEL_DSI_DSI               =  2,       /* DSI controls 'out' and 'output enable' */
    HSIOM_SEL_DSI_GPIO              =  3,       /* DSI controls 'out', GPIO controls 'output enable' */
    HSIOM_SEL_AMUXA                 =  4,       /* Analog mux bus A */
    HSIOM_SEL_AMUXB                 =  5,       /* Analog mux bus B */
    HSIOM_SEL_AMUXA_DSI             =  6,       /* Analog mux bus A, DSI control */
    HSIOM_SEL_AMUXB_DSI             =  7,       /* Analog mux bus B, DSI control */
    HSIOM_SEL_ACT_0                 =  8,       /* Active functionality 0 */
    HSIOM_SEL_ACT_1                 =  9,       /* Active functionality 1 */
    HSIOM_SEL_ACT_2                 = 10,       /* Active functionality 2 */
    HSIOM_SEL_ACT_3                 = 11,       /* Active functionality 3 */
    HSIOM_SEL_DS_0                  = 12,       /* DeepSleep functionality 0 */
    HSIOM_SEL_DS_1                  = 13,       /* DeepSleep functionality 1 */
    HSIOM_SEL_DS_2                  = 14,       /* DeepSleep functionality 2 */
    HSIOM_SEL_DS_3                  = 15,       /* DeepSleep functionality 3 */
    HSIOM_SEL_ACT_4                 = 16,       /* Active functionality 4 */
    HSIOM_SEL_ACT_5                 = 17,       /* Active functionality 5 */
    HSIOM_SEL_ACT_6                 = 18,       /* Active functionality 6 */
    HSIOM_SEL_ACT_7                 = 19,       /* Active functionality 7 */
    HSIOM_SEL_ACT_8                 = 20,       /* Active functionality 8 */
    HSIOM_SEL_ACT_9                 = 21,       /* Active functionality 9 */
    HSIOM_SEL_ACT_10                = 22,       /* Active functionality 10 */
    HSIOM_SEL_ACT_11                = 23,       /* Active functionality 11 */
    HSIOM_SEL_ACT_12                = 24,       /* Active functionality 12 */
    HSIOM_SEL_ACT_13                = 25,       /* Active functionality 13 */
    HSIOM_SEL_ACT_14                = 26,       /* Active functionality 14 */
    HSIOM_SEL_ACT_15                = 27,       /* Active functionality 15 */
    HSIOM_SEL_DS_4                  = 28,       /* DeepSleep functionality 4 */
    HSIOM_SEL_DS_5                  = 29,       /* DeepSleep functionality 5 */
    HSIOM_SEL_DS_6                  = 30,       /* DeepSleep functionality 6 */
    HSIOM_SEL_DS_7                  = 31,       /* DeepSleep functionality 7 */

As well as the pin by pin definitions… like this for P0_2

    /* P0.2 */
    P0_2_GPIO                       =  0,       /* GPIO controls 'out' */
    P0_2_AMUXA                      =  4,       /* Analog mux bus A */
    P0_2_AMUXB                      =  5,       /* Analog mux bus B */
    P0_2_AMUXA_DSI                  =  6,       /* Analog mux bus A, DSI control */
    P0_2_AMUXB_DSI                  =  7,       /* Analog mux bus B, DSI control */
    P0_2_TCPWM0_LINE1               =  8,       /* Digital Active - tcpwm[0].line[1]:0 */
    P0_2_TCPWM1_LINE1               =  9,       /* Digital Active - tcpwm[1].line[1]:0 */
    P0_2_CSD_CSD_TX                 = 10,       /* Digital Active - csd.csd_tx:2 */
    P0_2_CSD_CSD_TX_N               = 11,       /* Digital Active - csd.csd_tx_n:2 */
    P0_2_LCD_COM2                   = 12,       /* Digital Deep Sleep - lcd.com[2]:0 */
    P0_2_LCD_SEG2                   = 13,       /* Digital Deep Sleep - lcd.seg[2]:0 */
    P0_2_SCB0_UART_RX               = 18,       /* Digital Active - scb[0].uart_rx:0 */
    P0_2_SCB0_I2C_SCL               = 19,       /* Digital Active - scb[0].i2c_scl:0 */
    P0_2_SCB0_SPI_MOSI              = 20,       /* Digital Active - scb[0].spi_mosi:0 */

If you look at the Cy_GPIO_Pin_Init function you will see that on line 96 it sets the register which picks the correct pin mux.

cy_en_gpio_status_t Cy_GPIO_Pin_Init(GPIO_PRT_Type *base, uint32_t pinNum, const cy_stc_gpio_pin_config_t *config)
{
    cy_en_gpio_status_t status = CY_GPIO_BAD_PARAM;

    if ((NULL != base) && (NULL != config))
    {
        uint32_t maskCfgOut;
        uint32_t tempReg;
        
        CY_ASSERT_L2(CY_GPIO_IS_PIN_VALID(pinNum));
        CY_ASSERT_L2(CY_GPIO_IS_VALUE_VALID(config->outVal));
        CY_ASSERT_L2(CY_GPIO_IS_DM_VALID(config->driveMode));
        CY_ASSERT_L2(CY_GPIO_IS_HSIOM_VALID(config->hsiom));  
        CY_ASSERT_L2(CY_GPIO_IS_INT_EDGE_VALID(config->intEdge)); 
        CY_ASSERT_L2(CY_GPIO_IS_VALUE_VALID(config->intMask));
        CY_ASSERT_L2(CY_GPIO_IS_VALUE_VALID(config->vtrip));
        CY_ASSERT_L2(CY_GPIO_IS_VALUE_VALID(config->slewRate));
        CY_ASSERT_L2(CY_GPIO_IS_DRIVE_SEL_VALID(config->driveSel));
        CY_ASSERT_L2(CY_GPIO_IS_VALUE_VALID(config->vregEn));
        CY_ASSERT_L2(CY_GPIO_IS_VALUE_VALID(config->ibufMode));
        CY_ASSERT_L2(CY_GPIO_IS_VALUE_VALID(config->vtripSel));
        CY_ASSERT_L2(CY_GPIO_IS_VREF_SEL_VALID(config->vrefSel));
        CY_ASSERT_L2(CY_GPIO_IS_VOH_SEL_VALID(config->vohSel));
    
        Cy_GPIO_Write(base, pinNum, config->outVal);
        Cy_GPIO_SetDrivemode(base, pinNum, config->driveMode);
        Cy_GPIO_SetHSIOM(base, pinNum, config->hsiom);

        Cy_GPIO_SetInterruptEdge(base, pinNum, config->intEdge);
        Cy_GPIO_SetInterruptMask(base, pinNum, config->intMask);
        Cy_GPIO_SetVtrip(base, pinNum, config->vtrip);

        /* Slew rate and Driver strength */
        maskCfgOut = (CY_GPIO_CFG_OUT_SLOW_MASK << pinNum) 
                     | (CY_GPIO_CFG_OUT_DRIVE_SEL_MASK << ((uint32_t)(pinNum << 1U) + CY_GPIO_CFG_OUT_DRIVE_OFFSET));
        tempReg = GPIO_PRT_CFG_OUT(base) & ~(maskCfgOut);
        
        GPIO_PRT_CFG_OUT(base) = tempReg | ((config->slewRate & CY_GPIO_CFG_OUT_SLOW_MASK) << pinNum)
                            | ((config->driveSel & CY_GPIO_CFG_OUT_DRIVE_SEL_MASK) << ((uint32_t)(pinNum << 1U) + CY_GPIO_CFG_OUT_DRIVE_OFFSET));

        /* SIO specific configuration */
        tempReg = GPIO_PRT_CFG_SIO(base) & ~(CY_GPIO_SIO_PIN_MASK);
        GPIO_PRT_CFG_SIO(base) = tempReg | (((config->vregEn & CY_GPIO_VREG_EN_MASK)
                                         | ((config->ibufMode & CY_GPIO_IBUF_MASK) << CY_GPIO_IBUF_SHIFT)
                                         | ((config->vtripSel & CY_GPIO_VTRIP_SEL_MASK) << CY_GPIO_VTRIP_SEL_SHIFT)
                                         | ((config->vrefSel & CY_GPIO_VREF_SEL_MASK)  << CY_GPIO_VREF_SEL_SHIFT)
                                         | ((config->vohSel & CY_GPIO_VOH_SEL_MASK) << CY_GPIO_VOH_SEL_SHIFT))
                                           << ((pinNum & CY_GPIO_SIO_ODD_PIN_MASK) << CY_GPIO_CFG_SIO_OFFSET));

        status = CY_GPIO_SUCCESS;
    }

    return(status);
}

And finally the Cy_GPIO_SetHSIOM actually writes to the register.

__STATIC_INLINE void Cy_GPIO_SetHSIOM(GPIO_PRT_Type* base, uint32_t pinNum, en_hsiom_sel_t value)
{
    uint32_t portNum;
    uint32_t tempReg;
    HSIOM_PRT_V1_Type* portAddrHSIOM;

    CY_ASSERT_L2(CY_GPIO_IS_PIN_VALID(pinNum));
    CY_ASSERT_L2(CY_GPIO_IS_HSIOM_VALID(value));

    portNum = ((uint32_t)(base) - CY_GPIO_BASE) / GPIO_PRT_SECTION_SIZE;
    portAddrHSIOM = (HSIOM_PRT_V1_Type*)(CY_HSIOM_BASE + (HSIOM_PRT_SECTION_SIZE * portNum));

    if(pinNum < CY_GPIO_PRT_HALF)
    {
        tempReg = HSIOM_PRT_PORT_SEL0(portAddrHSIOM) & ~(CY_GPIO_HSIOM_MASK << (pinNum << CY_GPIO_HSIOM_OFFSET));
        HSIOM_PRT_PORT_SEL0(portAddrHSIOM) = tempReg | ((value & CY_GPIO_HSIOM_MASK) << (pinNum << CY_GPIO_HSIOM_OFFSET));
    }
    else
    {
        pinNum -= CY_GPIO_PRT_HALF;
        tempReg = HSIOM_PRT_PORT_SEL1(portAddrHSIOM) & ~(CY_GPIO_HSIOM_MASK << (pinNum << CY_GPIO_HSIOM_OFFSET));
        HSIOM_PRT_PORT_SEL1(portAddrHSIOM) = tempReg | ((value & CY_GPIO_HSIOM_MASK) << (pinNum << CY_GPIO_HSIOM_OFFSET));
    }
}

PSoC 6 SPI

Finally, it may seem obvious, but there is a limited set of connections to each GPIO in PSoC 6.  This means that any given SCB can only connect its SPI pins to a specific set of pins on the chip.  But, what are they?  You can either look at the data sheet, or you can search the file which you will find the pin definitions which will all be in the form of Px_y_SCBz_SPI_MOSI and this will give you a complete map.

Stupid Python Tricks – Ensure PIP & Virtual Environments

Summary

This article will show you how to fix your Python setup such that virtual environments that you create will have the correct version of PIP

The Story

I frequently take screen shots as part of my article writing process.  And it absolutely drives me crazy to have “crap” on the screen e.g. warning messages.  I was recently working on an article about PyVISA – a Python package for interacting with Lab Instruments – when I got this error message about the wrong PIP version.

But how can this be as I know that I have the correct version (which at the time was 20.0.2).  You obviously can “fix” this by running an upgrade of pip in your virtual environment.

But that doesn’t really “fix” it.  It just means that the virtual environment you just created has the most up to date PIP.  What is frustrating is that when you exit the virtual environment, the PIP version is correct (look 20.0.2)

Why are the environments differ?   The answer is when you create a virtual environment it will copy Python, pip and easy_install to your binary directory.

As part of doing this it will pickup the “ensurepip” version of PIP which is embedded in the Python installation on your computer.  Ensure pip just ensures that there will be a pip version available in any given Python installation even though Pip is not installed with Python.  On my computer the ensurepip is embedded deeply in a Hombrew directory:

In the screen above you can see that my ensurepip has “19.2.3” when the rest of my computer is set to “20.0.2”. So how do you fix the ensurepip?  Well I first thought that perhaps running the “ensurepip” with the upgrade flag would do it.  Here is what happens.  It seems to fix it.  (but it doesn’t).

The only way that I know how to fix it is install the Python module upgrade_ensurepip.  Here are the commands

  • pip3 install upgrade_ensurepip
  • python3 -m upgrade_ensurepip

Now you when create a virtual environment you will end up with the correct PIP

If you look in the directory you will find that it adds the “pip… whl” director into the ensurepip directory

And it modifies the “_PIP_VERSION” in the ensurepip package __init.py

Keithley 2380-500-15 & 2380-120-60

Summary

In this article I discuss my ridiculous motivation for buying a new Keithley 2830-120-60 to replace my very functional Keithey 2380-500-15

2380-500-15 & 2380-120-60

While working on the IRDC3894 I spent a significant amount of time using the Keithley 2380 in Constant Current Mode.  The development kit that I was testing was configured to enable 1.2v output at 15A.  In the picture below I am pulling 1A.  You can see the Keithley set to pull 1A and it is actually drawing 0.9998A (plenty close)

While I was testing the setup I would slowly increase the current.  In the picture below you can see that I got to 5.4A with no problem.

But at 5.5A the trouble starts.  In the picture below you can see that I am asking for 5.5A but I am only getting 5.48A

And the gap gets worse as I increase the current.

So I posted on the Keithley site trying to figure out what was happening.  Was the Keithley dead?

And unfortunately there is the answer.  The load has a minimum operating voltage of 4.5v when it is in the 15A mode.

But the 2380-120-60 has a 1.8V operating voltage at 60A

And when I get it plugged in I find that it will happily deliver 16A at 1.2V

And it doesn’t start to roll over until 17A (at 1.2V)

PSoC 6 SDK OneWire Bus (Part 4): But, Can It Read the Temperature?

Summary

In this series of articles I have been implementing a one-wire sensor library to work with PSoC 6.  In this article I will test the library by reading the temperature from the DS18B20 sensor.

Story

After three articles crawling through one-wire documents and firmware we are down to the real question.  Will it actually read the temperature?  Each device on a one-wire bus is addressed by a unique 64-bit value which is programmed at the factory.  Before you can talk to a device you need to read the rom value.  The question is how do you find the ROM value when you have multiple devices connected on the bus?  There are two ways

  1. Follow a rather complicated discovery process (next article)
  2. If there is only 1 device on the bus you can follow a short cut.

Well, lets start with the simple method.

Read ROM

The data sheet is pretty clear about how to read the ROM code when you only have one device.  Basically, you send a 0x33, then you read 8 bytes.  Here is a clip from the data sheet.

I will add a command to my ntshell command line.  The command is “readrom” it will

  1. Send the reset
  2. Wait 1MS (I don’t think you have to do this, but this was a bug work around)
  3. Send the 0x33
  4. Then read 8 bytes.
  5. And print out the result (lines
OneWireBus_ROMCode rom;

static int usrcmd_readrom(int argc, char **argv)
{
    owb_ret_t rval;

    printf("Sending reset\n");
    bool result;
    rval = owb_reset(&bus,&result);
    if(rval != OWB_STATUS_OK)
    {
        printf("Reset failed\n");
        return 0;
    }

    CyDelay(1);

    printf("Sending 0x33\n");

    rval = owb_write_byte(&bus,0x33);
    if(rval != OWB_STATUS_OK)
    {
        printf("Write 0x33 failed\n");
        return 0;
    }


    // read 64-bit rom
    printf("Reading 8 bytes\n");

    rval = owb_read_bytes(&bus,rom.romAddr,8);

   if(rval != OWB_STATUS_OK)
    {
        printf("read 8 bytes failed\n");
        return 0;
    }
    // 
    printf("Rom = ");
    for(int i=0;i<8;i++)
    {
        printf("%02X ",rom.romAddr[i]);
    }
    printf("\n");
    return 0;
}

When I test it, things seem to be working.

Read Temperature

Now that we know the ROM Code, which is used as the address, how do we get the temperature.  To do this you should follow this procedure.

  1. Do a Match ROM
  2. Send the ROM address
  3. Send a Convert Temperature
  4. Wait for the right amount of time
  5. Send a Read Scratch Pad
  6. Read 9 Bytes
  7. Convert the values

The Match ROM command 0x55 + the ROM code selects your device to be acted on.

When you send a convert T 0x44 your device will start the internal process of reading the temperature and storing it in the scratch pad.  The amount of time this take depends on what resolution you have configured.

The scratchpad is a 9-byte register inside of the chip which contains the most recently read temperature plus some settings.

To read the scratchpad you need to send a 0xBE, then read 9-bytes.  Notice that the 9th byte is a CRC for the first 8 bytes,  if you are concerned you can follow the CRC procedure documented in the datasheet to calculate a CRC to verify a match.  For this example, lets skip that.

In order to convert the temperature you need to know the resolution.  Bit-6 and Bit-5 of the configuration register tells you this.  This is a writable register so you can change the resolution which you might do to speed up the reading time.

Here is the whole code together as a new command.

static int usrcmd_readtemp(int argc, char **argv)
{
    uint8_t scratchpad[9];
    bool result;

    // send reset
    // send match rom 0x55
    // send rom address
    // send trigger 0x44
    // delay 1 second
    // send reset
    // send match rom 0x55
    // send read scratchpad BEh
    // read 9 bytes of scratch pad

    owb_reset(&bus,&result);
    owb_write_byte(&bus,0x55);
    owb_write_bytes(&bus,rom.romAddr,8);
    owb_write_byte(&bus,0x44);
    vTaskDelay(1000);
    owb_reset(&bus,&result);

    owb_write_byte(&bus,0x55);
    owb_write_bytes(&bus,rom.romAddr,8);
    owb_write_byte(&bus,0xbe);
    owb_read_bytes(&bus,scratchpad,9);
    printf("Scratchpad =");
    for(int i=0;i<9;i++)
    {
        printf("%02X ",scratchpad[i]);
    }
    printf("\n");
 
    int16_t tempbits = scratchpad[0] | (scratchpad[1] << 8);
    uint32_t resolution=2^12;
    float temperature=0.0;
    switch(scratchpad[5] >>5 & 0x03)
    {
        case 0:
            resolution = 2^9;
            break;
        case 1:
            resolution = 2^10;
            break;
        case 2:
            resolution = 2^11;
            break;
        case 3:
            resolution = 2^12;
        break;
    }

    temperature = (float)tempbits / (float)resolution;

    printf("Temperature = %f\n",temperature);
    return 0;  

}

Now I build and program the development kit to setup the test.  The first step is to read the ROM with the command line “readrom”.  Then I tell it to try to read the temperature.  You can see the values in the scratch pad, plus my conversion to celsius.  Since I am in Kentucky I probably should have done Fahrenheit … oh well. 28.35C is 83.3F my office is pretty hot today.

PSoC 6 SDK OneWire Bus (Part 3): Remove Busy Wait & Debug

Summary

In this article I will replace the stupid busy wait implementation of OneWire write and read bit with an interrupt based implementation.

Story

In the previous article, I implemented a OneWire bus library for PSoC 6.  Specifically, to read temperatures from the DS18B20 1-wire temperature sensor.  This implementation used the CyDelay to handle bit timing. Here is the write bit function.

owb_ret_t owb_write_bit(OneWireBus *bus,uint8_t val)
{
    owb_ret_t rval = OWB_STATUS_OK;

    cyhal_gpio_write(bus->pin,0);
    if(val == 0)
    {
        CyDelayUs(60);
        cyhal_gpio_write(bus->pin,1);
        CyDelayUs(2);
    }
    else
    {
        CyDelayUs(1);
        cyhal_gpio_write(bus->pin,1);
        CyDelayUs(60);
    }
    return rval;
}

And the read bit function.

owb_ret_t owb_read_bit( OneWireBus *bus, uint8_t *bit)
{
    owb_ret_t rval = OWB_STATUS_OK;

    cyhal_gpio_write(bus->pin,0);
    CyDelayUs(1);
    cyhal_gpio_write(bus->pin,1);
    CyDelayUs(5);

    *bit = cyhal_gpio_read(bus->pin);
    CyDelayUs(60);
    return rval;
}

Both of these functions APPEAR to work perfectly.  Here is a write of 0xFF

However, both of these implementations are subject to fail if an interrupt from the RTOS occurs during the delay period.  You could easily end up with a delay that is too long by enough that the sensor attached to the bus won’t work.  Moreover, I have always hated busy wait loops as the processor could be doing something useful.  So let’s replace this with something better.

This is a late edit to the article, but as I read the documentation for the “owb” library I found this note in Dave Antliff’s documentation

Im not exactly sure what a “RMT” is in the world of ESP32, but I gotta imagine that it is a hardware timer.

Implement an HW Timer and Interrupt Scheme

Both the read and write operation require that the master

  • Write a 0
  • Wait for some time
  • Write a 1
  • Wait for some time (then optionally read)
  • Wait for some time end

The PSoC 6 TCPWM can be used to generate accurate interrupts at specific times in the future.   This hardware is abstracted using the Cypress HAL which will let you configure

  • The input frequency to the timer
  • The counter in the timer
  • A “compare” value to trigger an interrupt with event type CYHAL_TIMER_IRQ_CAPTURE_COMPARE
  • A “period” value to trigger an interrupt with the event type CYHAL_TIMER_IRQ_TERMINAL_COUNT

If you are confused about the TCPWM hardware you are not alone, it can do a mind blowing amount of different things.  However in this simple case we have it set to

  1. Input frequency 1 Mhz (aka 1uS per count)
  2. Start at 0
  3. Trigger an event CYHAL_TIMER_IRQ_CAPTURE_COMPARE at the compare value
  4. Tigger an event CYHAL_TIMER_IRQ_TERMINAL_COUNT at the period
  5. Stop counting
  6. Reset back to 0

If you look the code below on lines 23-30 the configuration of the timer is set.  Remember from the data sheet that a “1” is a 1uS 0 followed by a 60uS 1 and a 0 is a 60uS 0 followed by a 1uS 1.  I use the “compare” to be the trigger from 0–>1.  Notice lines 5-9 in the code below.

On lines 40-41 I tell the HAL that I am interested in getting an event interrupt.

Line 43 initiates the write with a write of 0.  Then line 45 starts the timer going.

To “end” the operation I use a semaphore to wait until the interrupt is triggered by the Terminal Count (aka Period).

void write_bit_event_callback(void *callback_arg, cyhal_timer_event_t event)
{
    OneWireBus *bus = (OneWireBus *)callback_arg;

    if(event & CYHAL_TIMER_IRQ_CAPTURE_COMPARE)
    {
        cyhal_gpio_write(bus->pin,1);
    }
    if(event & CYHAL_TIMER_IRQ_TERMINAL_COUNT)
    {
        BaseType_t xHigherPriorityTaskWoken;
        xHigherPriorityTaskWoken = pdFALSE;
        xSemaphoreGiveFromISR(bus->signalSemaphore,&xHigherPriorityTaskWoken);
        portYIELD_FROM_ISR( xHigherPriorityTaskWoken );
    }

}

owb_ret_t owb_write_bit(OneWireBus *bus,uint8_t val)
{
    owb_ret_t rval;

    cyhal_timer_cfg_t cfgBit = {
        .is_continuous = false,
        .direction = CYHAL_TIMER_DIR_UP,
        .is_compare = true,
        .compare_value = 1,
        .period = 70,
        .value = 0,
    };

    if(val == 0)
        cfgBit.compare_value = 65;
    else
        cfgBit.compare_value = 1;
    
    cyhal_timer_configure(&bus->bitTimer,&cfgBit);
    cyhal_timer_reset(&bus->bitTimer);

    cyhal_timer_register_callback(&bus->bitTimer,write_bit_event_callback,bus);
    cyhal_timer_enable_event(&bus->bitTimer,CYHAL_TIMER_IRQ_TERMINAL_COUNT|CYHAL_TIMER_IRQ_CAPTURE_COMPARE,5,true);

    cyhal_gpio_write(bus->pin,0);

    cyhal_timer_start(&bus->bitTimer);
    
    BaseType_t rsem = xSemaphoreTake(bus->signalSemaphore,1); 

    if(rsem == pdTRUE)
        rval = OWB_STATUS_OK;
    else
        rval = OWB_STATUS_ERROR;

    return rval;
}

Fixing the Write Bug

When I programmed this code I got some weird stuff.  Suck.  So to debug it, I put in a command line to let me “wbyte ff” (or whatever byte value).  At the top of usrcmd.c you need to declare the usrcmd_wbyte function and add it to the command table (notice that I chopped out everything around it)

static int usrcmd_wbyte(int argc, char **argv);

static const cmd_table_t cmdlist[] = {

    { "wbyte","write byte", usrcmd_wbyte},

};

Then make a function will will take an argument (the value you want to write).  Notice that the input is a 2 digit hex number.  Yes I known sscanf is dangerous (but in this case I use it only as a debugging tool.)

static int usrcmd_wbyte(int argc, char **argv)
{
    if(argc != 2)
    {
        printf("Invalid argument write  %d\n",argc);
        return 0;
    }

    owb_ret_t ret=OWB_STATUS_OK;

    int val;
    sscanf(argv[1],"%02x",&val);

    printf("Write Byte Val = %02X\n",val);

    ret = owb_write_byte(&bus,(uint8_t)val);

    if(ret == OWB_STATUS_OK)
        printf("Write byte succeeded\n");
    else
        printf("Write byte failed\n");


    return 0;
}

When I run the code I seem to get the write byte function working sometimes and failing sometimes.  What the hell?

Now, the pain starts.  To debug, I get out the oscilloscope and have a look.  Here is a write of 0xFF which should be 8 short 0 pulses.  But notice that only three get written.  OK, that must mean that the semaphore is timing out. How can that be?

To try to debug that I update a pin to toggle with the semaphore (and git rid of the exit case).

    cyhal_gpio_write(CYBSP_D5,1);
    BaseType_t rsem = xSemaphoreTake(bus->signalSemaphore,1); 
    cyhal_gpio_write(CYBSP_D5,0);

Now I get this (on the debug pin)… notice a very short timeout… then longer ones followed by another partially shorter timeout.

The problem is the semaphore timeout is set to 1.  Should be at least 1 ms right?  Why is that a problem as 1ms is way more than the 60-ish uS you need to wait?

BaseType_t rsem = xSemaphoreTake(bus->signalSemaphore,1);

Classic off by 1 error.  The 1 means expire at the NEXT SysTick, which will happen NO MORE than 1ms from now. Change it to 2.

    BaseType_t rsem = xSemaphoreTake(bus->signalSemaphore,2); 

Now, all the bits come out at a steady rate.

And it works… sometimes…

Deep Sleep

Other times I end up with this picture.  Notice that the space between the start of one bit and the start of the next bit (from a-b on the scope) is 361.6 uS (don’t forget the 0.6 uS).

Why are the interrupts getting delayed?  The answer is the chip is in deep sleep (or going to deep sleep) when the interrupt from the timer happens.  And when the deep sleep is happening, it takes a bit of time to go to deep sleep, then to wake up.  To fix this go into the system configurator and change the Deep Sleep Latency.

Change the timeout to 3

Now it works.. here is a write of 0xFE

Implement the Read Bit

Now that the write bit is working, move the read bit function to the same scheme.  Notice that I use the compare to trigger the read of the pin.  In the read code I probably should have done a critical section from the start of the 0 to the start of the timer so that an interrupt doesn’t occur before the timer gets started.

void read_bit_event_callback(void *callback_arg, cyhal_timer_event_t event)
{
    OneWireBus *bus = (OneWireBus *)callback_arg;

    if(event & CYHAL_TIMER_IRQ_CAPTURE_COMPARE)
    {
        bus->scratchBitValue = cyhal_gpio_read(bus->pin);
    }
    if(event & CYHAL_TIMER_IRQ_TERMINAL_COUNT)
    {
        BaseType_t xHigherPriorityTaskWoken;
        xHigherPriorityTaskWoken = pdFALSE;
        xSemaphoreGiveFromISR(bus->signalSemaphore,&xHigherPriorityTaskWoken);
        portYIELD_FROM_ISR( xHigherPriorityTaskWoken );
    }

}


owb_ret_t owb_read_bit( OneWireBus *bus, uint8_t *bit)
{
    owb_ret_t rval;

    cyhal_timer_cfg_t cfgBit = {
        .is_continuous = false,
        .direction = CYHAL_TIMER_DIR_UP,
        .is_compare = true,
        .compare_value = 10,
        .period = 61,
        .value = 0
    };

    cyhal_timer_configure(&bus->bitTimer,&cfgBit);

    cyhal_timer_register_callback(&bus->bitTimer,read_bit_event_callback,bus);
    cyhal_timer_enable_event(&bus->bitTimer,CYHAL_TIMER_IRQ_TERMINAL_COUNT|CYHAL_TIMER_IRQ_CAPTURE_COMPARE,5,true);

    // Pull a 0
    cyhal_gpio_write(bus->pin,0);
    CyDelayUs(1);
    cyhal_gpio_write(bus->pin,1);

    cyhal_timer_reset(&bus->bitTimer);
    cyhal_timer_start(&bus->bitTimer);

    BaseType_t rsem = xSemaphoreTake(bus->signalSemaphore,2); // Then entire write cycle is 61uS so 2ms would be a major failure
    *bit = bus->scratchBitValue;

    if(rsem == pdTRUE)
        rval = OWB_STATUS_OK;
    else
        rval = OWB_STATUS_ERROR;

    return rval;
}

In fact as I look at this code, I decide that I had better to fix the critical section.  Here is the updated read code.

    void taskENTER_CRITICAL();

    cyhal_gpio_write(bus->pin,0);     // Pull a 0
    CyDelayUs(1);
    cyhal_gpio_write(bus->pin,1);

    cyhal_timer_start(&bus->bitTimer);
    void taskEXIT_CRITICAL( );

And the updated write code

    taskENTER_CRITICAL();
    cyhal_gpio_write(bus->pin,0);

    cyhal_timer_start(&bus->bitTimer);
    taskEXIT_CRITICAL();

In the next article I’ll show you code to actually read the temperature.

 

PSoC 6 SDK OneWire Bus (Part 2) : Implement Read & Write

Summary

In this article I will explain how to implement (badly) the one wire bus read & write functions using the PSoC 6 SDK.  This is one of many articles that are part of a series discussing the implementation:

Story

In the previous article, I built a test framework inside of a PSoC 6 FreeRTOS project.  This includes a command line shell which will enable me to execute functions based on typed commands.  This means I can use the CLI shell to debug my interface.  To get going testing, I bought a DS18B20 temperature sensor from Mouser.

Then wired it like this:

Here is a picture of the test jig.  It looks like I used 4200 ohms instead of 4400 ohms, oh well it seems to work.

Init

Init seems like a good place to start.  To have a one wire bus you need to have a GPIO to read/drive.  So, that will be the only argument to the initialization function.  In the implementation I will use a common semaphore to handle blocking functions (line 13).  On lines 15-17 I initialize a timer to handle the requirements of reset, read and writes (more on this later).  At the end on line 20 I initialize the GPIO.

#include "owb.h"
#include <stdint.h>
#include "cyhal.h"
#include "cybsp.h"
#include "task.h"
#include <stdbool.h>
#include <stdio.h>

owb_ret_t owb_init(OneWireBus *bus)
{
    cy_rslt_t rslt;

    bus->signalSemaphore = xSemaphoreCreateBinary();

    rslt = cyhal_timer_init(&bus->bitTimer,NC,0);
    CY_ASSERT(rslt == CY_RSLT_SUCCESS);
    rslt = cyhal_timer_set_frequency(&bus->bitTimer,1000000);
    
    CY_ASSERT(rslt == CY_RSLT_SUCCESS);
    rslt = cyhal_gpio_init(bus->pin,CYHAL_GPIO_DIR_BIDIRECTIONAL,CYHAL_GPIO_DRIVE_OPENDRAINDRIVESLOW,true);
    CY_ASSERT(rslt == CY_RSLT_SUCCESS);
    return OWB_STATUS_OK;    
}

In the usrcmd.c I need to add includes for the new stuff.

#include "cybsp.h"
#include "owb.h"

The next step is to add the “init” command to usrcmd.c  This is accomplished by adding a function header for usrcmd_init and adding it to the table of legal commands.

static int usrcmd_init(int argc, char **argv);

typedef struct {
    char *cmd;
    char *desc;
    USRCMDFUNC func;
} cmd_table_t;

static const cmd_table_t cmdlist[] = {
    { "help", "This is a description text string for help command.", usrcmd_help },
    { "info", "This is a description text string for info command.", usrcmd_info },
    { "clear", "Clear the screen", usrcmd_clear },
    { "printargs","print the list of arguments", usrcmd_printargs},
    { "init","Initialize the 1-wire bus", usrcmd_init},

};

Then you need to add the usrcmd_init function to call the owb init.

static OneWireBus bus;
static int usrcmd_init(int argc, char **argv)
{
    bus.pin = CYBSP_D4;
    owb_init(&bus);
    printf("Initialized D4\n");

    return 0;
}

Now test it.  All good.

Reset

In order to reset the bus you follow this process

  1. Pull down the bus for trstl=480uS
  2. Let the bus go back to 1 (remember that it is restive pull-up)
  3. Wait another trsth=480uS
  4. If there is a slave device on the bus it will pull down the bus at some point to indicate a “presence detect”

From the data sheet you can see that the low& high times = 480uS.  At some point during the high period the slave will pull the bus down for 15->60uS

In order to implement the reset I will modify the owb.c file.  To meet the timing requirements I will use a cyhal_timer.  The cyhal_timer is implemented in the Cypress PDL at a Cy_TCPWM_Timer which is then implemented in the hardware as a timer counter.  In this configuration the timer has:

  • A Counter (an up or down counter)
  • A Compare (a register which can be compared with the counter to trigger events)
  • A Period (a register which is compared to the counter to reset the counter back to 0).   This event is also called the “terminal count”

For this implementation I will setup a timer with

  1. Input clock 1Mz (aka 1uS per count)
  2. An up counter
  3. Starts at 0
  4. Compare value = 480 (means trigger an interrupt at 480uS)
  5. Period = 960 (means trigger an interrupt at 480uS)
  6. One shot (stop counting when you get to period (aka Terminal Count))

The code will

  1. Configure the timer (line 41-52)
  2. Pull a 0 onto the bus (line 54)
  3. Start the timer (line 56)
  4. At the compare value 480uS reset the bus to 1 using the event handler (line 20) and setup the GPIO trigger a fall event interrupt (line 21-22)
  5. At the period release the semaphore (27-30)
  6. After the compare value if the GPIO pulls down, use the GPIO interrupt to record the presence of a device (line 10)
/////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Reset
/////////////////////////////////////////////////////////////////////////////////////////////////////////////
static void reset_gpio_event_callback(void *callback_arg, cyhal_gpio_event_t event)
{
    OneWireBus *bus = (OneWireBus *)callback_arg;

    if(event & CYHAL_GPIO_IRQ_FALL)
    {
        bus->detect = true;
    }
}

static void reset_timer_event_callback(void *callback_arg, cyhal_timer_event_t event)
{
    OneWireBus *bus = (OneWireBus *)callback_arg;

    if(event & CYHAL_TIMER_IRQ_CAPTURE_COMPARE)
    {
        cyhal_gpio_write(bus->pin,1);
        cyhal_gpio_register_callback(bus->pin, reset_gpio_event_callback,(void *)bus);
        cyhal_gpio_enable_event(bus->pin,CYHAL_GPIO_IRQ_FALL,5,true);
    }
    if(event & CYHAL_TIMER_IRQ_TERMINAL_COUNT)
    {

        BaseType_t xHigherPriorityTaskWoken;
        xHigherPriorityTaskWoken = pdFALSE;
        xSemaphoreGiveFromISR(bus->signalSemaphore,&xHigherPriorityTaskWoken);
        portYIELD_FROM_ISR( xHigherPriorityTaskWoken );
    }
 
}

owb_ret_t owb_reset(OneWireBus *bus, bool *result)
{
    owb_ret_t rval = OWB_STATUS_OK;

    bus->detect = false;

    cyhal_timer_cfg_t cfgBit = {
        .is_continuous = false,
        .direction = CYHAL_TIMER_DIR_UP,
        .is_compare = true,
        .compare_value = 480,
        .period = 960,
        .value = 0
    };
    cyhal_timer_configure(&bus->bitTimer,&cfgBit);
    cyhal_timer_reset(&bus->bitTimer);
    cyhal_timer_register_callback(&bus->bitTimer,reset_timer_event_callback,bus); 
    cyhal_timer_enable_event(&bus->bitTimer,CYHAL_TIMER_IRQ_ALL,5,true);

    cyhal_gpio_write(bus->pin,0);

    cyhal_timer_start(&bus->bitTimer);

    xSemaphoreTake(bus->signalSemaphore,2); // worst case is really 960uS
    cyhal_gpio_enable_event(bus->pin,CYHAL_GPIO_IRQ_FALL,5,true);
    return rval;
}

Once I have the owb_init code I add the init command to usrcmd.c.

static int usrcmd_reset(int argc, char **argv)
{
    bool result;
    owb_reset(&bus,&result);
    if(bus.detect)
    {
        printf("Reset Succeeded\n");

    }
    else

        printf("Reset Failed\n");

    return 0;
}

When I run it, I get this nice picture from the oscilliscope.

Write

In the ds18b20.c file I see that the author has three functions of interest.  In the implementation owb_write_bytes will call owb_write byte and owb_write_byte will call owb_write bit.

owb_ret_t owb_write_bit( OneWireBus *bus, uint8_t val);
owb_ret_t owb_write_byte( OneWireBus *bus, uint8_t val);
owb_ret_t owb_write_bytes( OneWireBus *bus, uint8_t *buffer, uint32_t length);

The one wire protocol has the following steps.

In order to write a “0” you need to

  1. Pull down the bus (write a 0)
  2. Wait 60uS
  3. Write a 1
  4. Wait 1uS (to allow the next “slot” to start

In order to write a “1” you need to

  1. Pull down the bus (write a 0)
  2. Wait 1 uS
  3. Write a 1
  4. Wait 60uS (to allow the next “slot” to start)

Notice that the minimum write slot is 60uS and the maximum is 120uS.  Here is a picture from the data sheet.

For the first implementation of owb_write_bit I will use a simple “CyDelayUs”  to implement the delays (this is a bad idea, but is a ‘cheap’ way to get going).

owb_ret_t owb_write_bit(OneWireBus *bus,uint8_t val)
{
    owb_ret_t rval = OWB_STATUS_OK;
    cyhal_gpio_write(bus->pin,0);
    if(val == 0)
    {
        CyDelayUs(60);
        cyhal_gpio_write(bus->pin,1);
        CyDelayUs(2);

    }
    else
    {
        CyDelayUs(1);
        cyhal_gpio_write(bus->pin,1);
        CyDelayUs(60);
    }
    return rval;
}

The write byte function just loops through the 8-bit and uses the owb_write_bit function.  Notice that it is least significant bit first.

owb_ret_t owb_write_byte( OneWireBus *bus, uint8_t val)
{
    owb_ret_t ret = OWB_STATUS_OK;
    for(int i=0;i<8;i++)
    {
        ret = owb_write_bit(bus,(val>>i) & 0x01);
        if(ret != OWB_STATUS_OK)
            return ret;        
    }
    return OWB_STATUS_OK;
}

And write_bytes (multiple) just loops through the buffer calling the write_byte function.

owb_ret_t owb_write_bytes( OneWireBus *bus, uint8_t *buffer, uint32_t length)
{
    owb_ret_t ret = OWB_STATUS_OK;
    for(int i=0;i<length;i++)
    {
        ret = owb_write_byte(bus,buffer[i]);
        if(ret != OWB_STATUS_OK)
            return ret;
    }
    return OWB_STATUS_OK;
}

Now that I have the three write functions I will add a new command to usrcmd.c  This function lets the user type a command like “wbyte a1” where he/she can input a 2-character hex command.  Yes, I use the sscanf function which is unsafe… but cheap for a test rig.

static int usrcmd_wbyte(int argc, char **argv)
{
    if(argc != 2)
    {
        printf("Invalid argument write  %d\n",argc);
        return 0;
    }

    owb_ret_t ret=OWB_STATUS_OK;

    int val;
    sscanf(argv[1],"%02x",&val);

    printf("Write Byte Val = %02X\n",val);

    ret = owb_write_byte(&bus,(uint8_t)val);

    if(ret == OWB_STATUS_OK)
        printf("Write byte succeeded\n");
    else
        printf("Write byte failed\n");


    return 0;
}

When I program and launch the project I end up here.

OK that is easy enough to fix, I just ran out of stack in the ntshell thread.

    xTaskCreate(ntShellTask, "nt shell task", configMINIMAL_STACK_SIZE*3,0 /* args */ ,0 /* priority */, 0);

Now when I run it I am off to the races.

Here is an example of “wbyte ff”

What about “owb_write_rom_code”?  I don’t know what it does, but Ill figure it out as I integrate the rest of the library.

Read

When I examine the ds18b20.c file I find these three read functions, which mirror the write functions.

owb_ret_t owb_read_bit( OneWireBus *bus, uint8_t *bit);
owb_ret_t owb_read_byte( OneWireBus *bus, uint8_t *byte);
owb_ret_t owb_read_bytes( OneWireBus *bus, uint8_t *buffer, uint32_t length);

To read a bit you follow a a very similar process to the write.

  1. Write a 0 on the bus
  2. Wait 1uS
  3. Write a 1 on the bus (release the bus)
  4. Wait 5uS
  5. Read (the slave will pull a 0 or a 1 onto the bus)
  6. Wait 55uS to the end of the slot

Here is a picture

The code uses a really bad busy-wait delay (but it is a good starting place to figure out what is happening)

owb_ret_t owb_read_bit( OneWireBus *bus, uint8_t *bit)
{
    owb_ret_t rval = OWB_STATUS_OK;

    cyhal_gpio_write(bus->pin,0);
    CyDelayUs(1);
    cyhal_gpio_write(bus->pin,1);
    CyDelayUs(5);

    *bit = cyhal_gpio_read(bus->pin);
    CyDelayUs(55);
    return rval;
}

To read a byte, just read 8 bits using the owb_read_bit function

owb_ret_t owb_read_byte( OneWireBus *bus, uint8_t *byte)
{
    uint8_t bit;
    owb_ret_t ret = OWB_STATUS_OK;

    *byte = 0;
    for(int i=0;i<8;i++)
    {
        ret = owb_read_bit(bus,&bit);
        if(ret != OWB_STATUS_OK)
            return ret;
        *byte = *byte | (bit<<i);
    }
    return OWB_STATUS_OK;
}

To read an array of bytes, use a loop of read_byte

owb_ret_t owb_read_bytes( OneWireBus *bus, uint8_t *buffer, uint32_t count)
{
    owb_ret_t ret = OWB_STATUS_OK;
    for(int i=0;i<count;i++)
    {
        ret = owb_read_byte(bus,&buffer[i]);
        if(ret != OWB_STATUS_OK)
            return ret;       
    }
    return ret;
}

Now that you have a read and a write, update the usrcmd.c to add a “rbyte” command to the CLI

static int usrcmd_rbyte(int argc, char **argv)
{

    owb_ret_t ret=OWB_STATUS_OK;

    uint8_t val;
 
    ret = owb_read_byte(&bus,&val);

    if(ret == OWB_STATUS_OK)
        printf("Read byte succeeded %02x\n",val);
    else
        printf("Read byte failed\n");


    return 0;
}

When I program the kit I can send a 0x33 (which will make the sensor respond more on this later), then read the first byte of the ROM

When you look at the scope picture (from right to left) you can see 10110110 which sure enough is B6.  The thin gaps are 1’s and the wide gaps are 0’s

In the next article Ill deal with the stupid CyDelay’s

PSoC 6 SDK OneWire Bus (Part 1) : Build Basic Project

Summary

This is the first article in a series about my journey implementing PSoC 6 SDK libraries for the Maxxim One Wire Bus and the DS18B20 temperature sensor.

As you can see there will be many parts to this story:

Story

I recently got a twitter message from a gentleman named “Neeraj Dhekale”.  He asked about a library for a one wire sensor for PSoC, actually to be specific he asked about a component for a PSoC 4.

Then I asked him what sensor and responded that he wanted to use the Maxxim DS18B20 temperature sensor.

This sensor is a one wire temperature sensor, here is a bit of snapshot from the data sheet.

After reading the data sheet I decided that I really didn’t want to implement a complete library for this sensor.  So, I started looking around for a driver library.  After googling around a little bit I found a library on GitHub (https://github.com/DavidAntliff/esp32-ds18b20) which looked promising, even though it is ESP32 specific.

But, after looking at this GitHub library a bit, I decide to start from there.

Before I get started two comments.

  1. He asked for PSoC 4 … but I am going to do PSoC 6 (because I can use Modus Toolbox).  There is no reason why this wouldn’t work on PSoC 4 – but it would take a bit of work
  2. I can’t think of a good reason to use 1-wire, it sure seems like I2C would be simpler

Build a Base Project

I start this whole effort by creating a new project for the CY8CKIT-062S2-43012 (because that happens to be the kit on my desk at the moment)

I will use the IoT Expert FreeRTOS Template project

For debugging this whole thing I will use a command line shell.  To get this into the project I add the ntshell library. Run “make modlibs” to start the library manager.

In the library manager pick the “ntshell” library

To develop this project I want to use Visual Studio code.  So, I run “make vscode” (to create the configuration files) and start vscode by running “code .”

When VSCODE starts up it looks like this:

In order to use the ntshell library you need to shuffle the files around a little bit.  Move the ntshell.h/.c into the main project by doing a drag/drop in the explorer window.

It will ask you really want to move the files

Once it is done, the ntshell functions which you need to customize will be part of the project.

To use the ntshell, you need to add the task to main.c

#include "cy_pdl.h"
#include "cyhal.h"
#include "cybsp.h"
#include "cy_retarget_io.h"
#include <stdio.h>
#include "FreeRTOS.h"
#include "task.h"

#include "ntshell.h"
#include "ntlibc.h"
#include "psoc6_ntshell_port.h"

// Global variable with a handle to the shell
ntshell_t ntshell;

void ntShellTask()
{

  printf("Started ntshell\n");
  setvbuf(stdin, NULL, _IONBF, 0);
  ntshell_init(
	       &ntshell,
	       ntshell_read,
	       ntshell_write,
	       ntshell_callback,
	       (void *)&ntshell);
  ntshell_set_prompt(&ntshell, "DS18B20> ");
  vtsend_erase_display(&ntshell.vtsend);
  ntshell_execute(&ntshell);
}

And start the task

    xTaskCreate(ntShellTask, "nt shell task", configMINIMAL_STACK_SIZE*2,0 /* args */ ,0 /* priority */, 0);

Build and compile

And you should have a working project.

Run “make program” to get the board going

arh (master *) OWB_DS18B20 $ make program
Tools Directory: /Applications/ModusToolbox/tools_2.1

Prebuild operations complete
Commencing build operations...

Tools Directory: /Applications/ModusToolbox/tools_2.1

Initializing build: mtb-example-psoc6-empty-app Debug CY8CKIT-062S2-43012 GCC_ARM

Auto-discovery in progress...
-> Found 202 .c file(s)
-> Found 50 .S file(s)
-> Found 27 .s file(s)
-> Found 0 .cpp file(s)
-> Found 0 .o file(s)
-> Found 4 .a file(s)
-> Found 450 .h file(s)
-> Found 0 .hpp file(s)
-> Found 0 resource file(s)
Applying filters...
Auto-discovery complete

Constructing build rules...
Build rules construction complete

==============================================================================
= Building application =
==============================================================================
Building 181 file(s)
==============================================================================
= Build complete =
==============================================================================

Calculating memory consumption: CY8C624ABZI-D44 GCC_ARM -Og

   ---------------------------------------------------- 
  | Section Name         |  Address      |  Size       | 
   ---------------------------------------------------- 
  | .cy_m0p_image        |  0x10000000   |  5972       | 
  | .text                |  0x10002000   |  49532      | 
  | .ARM.exidx           |  0x1000e17c   |  8          | 
  | .copy.table          |  0x1000e184   |  24         | 
  | .zero.table          |  0x1000e19c   |  8          | 
  | .data                |  0x080022e0   |  1688       | 
  | .cy_sharedmem        |  0x08002978   |  8          | 
  | .noinit              |  0x08002980   |  148        | 
  | .bss                 |  0x08002a14   |  985176     | 
  | .heap                |  0x080f3270   |  46480      | 
   ---------------------------------------------------- 

  Total Internal Flash (Available)          2097152    
  Total Internal Flash (Utilized)           59468      

  Total Internal SRAM (Available)           1046528    
  Total Internal SRAM (Utilized)            1033500    


Programming target device... 
Open On-Chip Debugger 0.10.0+dev-3.0.0.665 (2020-03-20-17:12)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "swd". To override use 'transport select <transport>'.
adapter speed: 2000 kHz
** Auto-acquire enabled, use "set ENABLE_ACQUIRE 0" to disable
cortex_m reset_config sysresetreq
cortex_m reset_config sysresetreq
Info : Using CMSIS loader 'CY8C6xxA_SMIF' for bank 'psoc6_smif0_cm0' (footprint 6485 bytes)
Warn : SFlash programming allowed for regions: USER, TOC, KEY
Info : CMSIS-DAP: SWD  Supported
Info : CMSIS-DAP: FW Version = 2.0.0
Info : CMSIS-DAP: Interface Initialised (SWD)
Info : SWCLK/TCK = 1 SWDIO/TMS = 1 TDI = 0 TDO = 0 nTRST = 0 nRESET = 1
Info : CMSIS-DAP: Interface ready
Info : KitProg3: FW version: 1.11.159
Info : KitProg3: Pipelined transfers disabled, please update the firmware
Warn : *******************************************************************************************
Warn : * KitProg firmware is out of date, please update to the latest version using fw-loader at *
Warn : * ModusToolbox/tools/fw-loader                                                            *
Warn : *******************************************************************************************
Info : VTarget = 3.263 V
Info : kitprog3: acquiring PSoC device...
Info : clock speed 2000 kHz
Info : SWD DPIDR 0x6ba02477
Info : psoc6.cpu.cm0: hardware has 4 breakpoints, 2 watchpoints
Info : psoc6.cpu.cm0: external reset detected
***************************************
** Silicon: 0xE402, Family: 0x102, Rev.: 0x11 (A0)
** Detected Device: CY8C624ABZI-S2D44A0
** Detected Main Flash size, kb: 2048
** Flash Boot version: 3.1.0.45
** Chip Protection: NORMAL
***************************************
Info : psoc6.cpu.cm4: hardware has 6 breakpoints, 4 watchpoints
Info : psoc6.cpu.cm4: external reset detected
Info : Listening on port 3333 for gdb connections
Info : Listening on port 3334 for gdb connections
Info : kitprog3: acquiring PSoC device...
target halted due to debug-request, current mode: Thread 
xPSR: 0x41000000 pc: 0x00000190 msp: 0x080ff800
** Device acquired successfully
** psoc6.cpu.cm4: Ran after reset and before halt...
target halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x0000012a msp: 0x080ff800
** Programming Started **
auto erase enabled
Info : Flash write discontinued at 0x10001754, next section at 0x10002000
Info : Padding image section 0 at 0x10001754 with 172 bytes (bank write end alignment)
[100%] [################################] [ Erasing     ]
[100%] [################################] [ Programming ]
Info : Padding image section 1 at 0x1000e844 with 444 bytes (bank write end alignment)
[100%] [################################] [ Erasing     ]
[100%] [################################] [ Programming ]
wrote 57856 bytes from file /Users/arh/proj/xxx/OWB_DS18B20/build/CY8CKIT-062S2-43012/Debug/mtb-example-psoc6-empty-app.hex in 2.294771s (24.621 KiB/s)
** Programming Finished **
** Verify Started **
verified 57240 bytes in 0.155447s (359.598 KiB/s)
** Verified OK **
** Resetting Target **
shutdown command invoked

arh (master *) OWB_DS18B20 $

And you should have a working project (with a blinking led and a command line)

Add the DS18B20 Library

Now that I have a working project the next step is to clone the DS18B20 library into my project.  This is done using

  • git clone https://github.com/DavidAntliff/esp32-ds18b20

The next step is to establish how bad things are with the new library.  So run the compiler.

When I look at the ds18b20.c file I can see that there are some obvious problems

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "driver/gpio.h"
#include "esp_system.h"
#include "esp_log.h"

#include "ds18b20.h"
#include "owb.h"

Fix those:

#include "FreeRTOS.h"
#include "task.h"
//#include "driver/gpio.h"
//#include "esp_system.h"
//#include "esp_log.h"

#include "ds18b20.h"
//#include "owb.h"

Run the compiler again.  Now I am missing owb.h which is the public header file for the one wire bus.

In file included from esp32-ds18b20/ds18b20.c:47:0:
./esp32-ds18b20/include/ds18b20.h:37:10: fatal error: owb.h: No such file or directory
 #include "owb.h"

I now make a directory to hold the new owb files called “p6sdk-onewire”.  Then I add a file “owb.h” (as a blank file)

When I run the compiler again, there are now two classes or error … owb and esp32 logging function. Here is an example of the ESP_ problems.

esp32-ds18b20/ds18b20.c:249:17: warning: implicit declaration of function 'ESP_LOG_BUFFER_HEX_LEVEL' [-Wimplicit-function-declaration]
                 ESP_LOG_BUFFER_HEX_LEVEL(TAG, scratchpad, count, ESP_LOG_DEBUG);
                 ^~~~~~~~~~~~~~~~~~~~~~~~
esp32-ds18b20/ds18b20.c:249:66: error: 'ESP_LOG_DEBUG' undeclared (first use in this function)
                 ESP_LOG_BUFFER_HEX_LEVEL(TAG, scratchpad, count, ESP_LOG_DEBUG);

And here is an example of the owb problems.

esp32-ds18b20/ds18b20.c: In function 'ds18b20_wait_for_conversion':
esp32-ds18b20/ds18b20.c:504:30: error: request for member 'use_parasitic_power' in something not a structure or union
         if (ds18b20_info->bus->use_parasitic_power)
                              ^~
esp32-ds18b20/ds18b20.c: At top level:
esp32-ds18b20/ds18b20.c:568:54: error: unknown type name 'OneWireBus'
 DS18B20_ERROR ds18b20_check_for_parasite_power(const OneWireBus * bus, bool * present)
                                                      ^~~~~~~~~~
esp32-ds18b20/ds18b20.c: In function 'ds18b20_check_for_parasite_power':
esp32-ds18b20/ds18b20.c:577:44: error: 'OWB_ROM_SKIP' undeclared (first use in this function); did you mean 'CY_ROM_SIZE'?
             if ((err = owb_write_byte(bus, OWB_ROM_SKIP)) == DS18B20_OK)
                                            ^~~~~~~~~~~~
                                            CY_ROM_SIZE
make[1]: *** [/Users/arh/proj/xxx/OWB_DS18B20/build/CY8CKIT-062S2-4301

To make this thing go, I edit ds18b20.c file and add logging templates (just stub functions that don’t do anything)

void ESP_LOGD(const char * code, char *val,...)
{

}

void ESP_LOGE(const char * code, ...)
{

}

void ESP_LOGW(const char *code,...)
{

}

#define ESP_LOG_DEBUG 0
void ESP_LOG_BUFFER_HEX_LEVEL(const char *code,...)
{

}

uint64_t esp_timer_get_time()
{
    return 0;
}

Create owb.h

If I was smart, I would have started with David Antliff “owb” library.  But I don’t.  What I do is search through “ds18b20.c” and find every function call to owb_ and then copy those function calls into owb.h (my new file).  Then I fix the function calls to have correct prototypes based on what I see in the ds18b20 library.  Here is what my owb.h file looks like after that process.

#ifndef OWB_H
#define OWB_H

#ifdef __cplusplus
extern "C" {
#endif

#define OWB_ROM_MATCH 0
#define OWB_ROM_SKIP 0

#include <stdint.h>
#include "cyhal.h"
#include "FreeRTOS.h"
#include "semphr.h"

typedef struct  
{
    cyhal_gpio_t  pin;
    bool use_parasitic_power;
    SemaphoreHandle_t signalSemaphore;
    cyhal_timer_t bitTimer;
    bool detect;
    SemaphoreHandle_t owb_num_active;
} OneWireBus;        

typedef struct {
    uint8_t romAddr[8];

} OneWireBus_ROMCode;

typedef enum {
    OWB_STATUS_OK,
    OWB_STATUS_ERROR,
} owb_ret_t ;

owb_ret_t owb_init(OneWireBus *bus);
    
owb_ret_t owb_reset(OneWireBus *bus, bool *result);

owb_ret_t owb_write_bit( OneWireBus *bus, uint8_t val);
owb_ret_t owb_write_byte( OneWireBus *bus, uint8_t val);
owb_ret_t owb_write_bytes( OneWireBus *bus, uint8_t *buffer, uint32_t length);

owb_ret_t owb_write_rom_code(OneWireBus *bus, OneWireBus_ROMCode romcode);

owb_ret_t owb_read_bit( OneWireBus *bus, uint8_t *bit);
owb_ret_t owb_read_byte( OneWireBus *bus, uint8_t *byte);
owb_ret_t owb_read_bytes( OneWireBus *bus, uint8_t *buffer, uint32_t length);

owb_ret_t owb_crc8_bytes(uint32_t val, uint8_t *buffer, uint32_t length);

void   owb_set_strong_pullup( OneWireBus *bus, bool val);

#endif

#ifdef __cplusplus
}
#endif

When I run the compiler again things look way better (just some complaining about const datatypes) and a complaint about the include path.  The include path thing is visual studio code not knowing that I added a new directory called p6sdk-onewire.

To fix the include path I run “make vscode” which tells VSCODE about the new directory.

That is a good place to split this article.  In the next article I will add functions to read and write the bus.

Keithley DAQ6510 & 7700

Summary

This article walks you through the first use of a Keithley 7700 20-channel multiplexer module attached to a Keithley DAQ6510.

Keithley describes the module in the data sheet as “The 7700 plug-in module offers 20 channels of 2-pole or 10 channels of 4-pole multiplexer switching that can be configured as two independent banks of multiplexers. There are two additional protected channels for current measurements. Automatic CJC
is provided so that no other accessories are required to make thermocouple temperature measurements. In addition, the 7700 contains latching electromechanical relays that enable signal bandwidths of up to 50 MHz. The 7700 is ideal for RTD, thermistor, and thermocouple temperature applications.”  And they give a nice picture:

And a “schematic”:

The Story

When I bought my original DAQ6510 from Mouser, they did not have a 7700 multiplexer module in stock.  So, I decided to buy one on eBay, which I was really hoping would work.  The module was salvaged out of some installation somewhere in California by a company called “Silicon Salvage”  I was a little bit worried about it because the multiplexor uses actual mechanical relays which wear out in somewhere between 100K and 100M switches.  That seemed like a lot, but who knows.

Assemble

When the unit arrived it seemed OK.  So I put my lab assistant to assembling and testing it.  To test it I bought a bunch of really inexpensive alligator to banana plug wires from China.

Then Nicholas clipped off the alligator ends and tinned the wires.  How about that classic soldering vice?  That was bought at an antique sale in Georgetown Kentucky a few years ago and works great for this kind of thing.  It is also heavy enough to kill Zombies with.

Then he installed the jumper wires onto the board.

Try it out

When everything was button up it was time to test.  Start with turning on the meter and pressing the rear button.

Then press “Build Scan”

Press the “Plus” symbol (to create a new list of channel and settings) to scan

Select some channels and press OK.  It turned out that we tested 3-wires at a time because I used a 3-channel power to supply to setup the voltages to test.

Then pick out DC Voltage

Press the Start Button to launch the meter to scan through the channels and save the values.

And the screen will look like this.

If you press the view scan status you will end on a screen like this.  Notice that you can only see channel 120.  To fix this press the “120”

Then select the other two channels

And you will now see all of the voltages

Here is a picture of the whole thing

Channel Grid App

The DAQ also have a function to display a grid of the channel values.  To get there Press the Apps button

Press “Channel Grid” then Run

It will then ask you to start the Scan

And when it is done you will have the voltages.

It would be really nice if this App had button to re-scan.  Or potentially a way to run the scans in a loop.  I am pretty sure that they give you a way to create Apps to run on this meter… so I suppose I’ll need to fix their App.

Reading Table

You can also view the scan data in a table.  To do this, press “Menu”

Press the “Reading Table”

Which will take you to see a table of the previous scan values.

Step Scan

You can also manually scan the channels by running a “Step Scan”.  Press the “Step Scan” button.

Which will read and display the first channel.

Then you can repeatedly click to work your way through all of the channels.

Stupid Python Tricks: VSCODE c_cpp_properties.json for Linux Kernel Development

Summary

This article shows you how to create a Python program that creates a Visual Studio Code c_cpp_properties.json which will enable intellisense when editing Linux Kernel Modules for Device Drivers.

Story

I am working my way through understanding, or at least trying to understand, Linux Kernel Modules – specifically Linux Device Drivers.  I have been following through the book Linux Device Drivers Development by John Madieu

There are a bunch of example codes in the book which you can “git”

$ git remote -v
origin  https://github.com/PacktPublishing/Linux-Device-Drivers-Development (fetch)
origin  https://github.com/PacktPublishing/Linux-Device-Drivers-Development (push)
$

When I started looking at the Chapter02 example I first opened it up in Visual Studio Code.  Where I was immediately given a warning about Visual Studio Code not knowing where to find <linux/init.h>

And, when you try to click on the “KERN_INFO” symboled you get this nice message.

In order to fix this you need to setup the “c_cpp_propertiese.json” to tell Visual Studio Code how to make intellisense work correctly. But, what is c_cpp_properties.json?

C_CPP_PROPERTIES.JSON

On the Visual Studio Code website they give you a nice description of the schema.  Here is a screenshot from their website.

The “stuff” that seems to matter from this list is the “includePath” which tells intellisense the #includes and #defines.  OK.  How do I figure out where to find all of the files and paths for that?

make –dry-run

When you look at the Makefile it doesn’t appear to help very much.  I don’t know about you guys, but I actually dislike “Make” way more than I dislike Python :-).  What the hell is this Makefile telling you to do?

On line 3 the Makefile creates a variable called “KERNELDIR” and sets the value to the result of the shell command “uname -r” plus “/lib/modules” at the first and “/build” at the end.  If I run the command “uname -r” on my system I get “4.15.0-99-generic”

Then on line 9 it calls make with a “-C” option

obj-m := helloworld-params.o helloworld.o

KERNELDIR ?= /lib/modules/$(shell uname -r)/build

all default: modules
install: modules_install

modules modules_install help clean:
	$(MAKE) -C $(KERNELDIR) M=$(shell pwd) $@

Which tells Make to “change directory”

$ make --help
Usage: make [options] [target] ...
Options:
  -b, -m                      Ignored for compatibility.
  -B, --always-make           Unconditionally make all targets.
  -C DIRECTORY, --directory=DIRECTORY
                              Change to DIRECTORY before doing anything

OK when I look in that directory I see a bunch of stuff.  Most importantly a Makefile.

$ cd /lib/modules/4.15.0-99-generic/build
$ ls
arch   crypto         firmware  init    Kconfig  Makefile        net      security  ubuntu
block  Documentation  fs        ipc     kernel   mm              samples  sound     usr
certs  drivers        include   Kbuild  lib      Module.symvers  scripts  tools     virt
$

When I look in that Makefile I start to sweat because it is pages and pages of Make incantations.  Now what?

# SPDX-License-Identifier: GPL-2.0
VERSION = 4
PATCHLEVEL = 15
SUBLEVEL = 18
EXTRAVERSION =
NAME = Fearless Coyote

# *DOCUMENTATION*
# To see a list of typical targets execute "make help"
# More info can be located in ./README
# Comments in this file are targeted only to the developer, do not
# expect to learn how to build the kernel reading this file.

# That's our default target when none is given on the command line
PHONY := _all
_all:

# o Do not use make's built-in rules and variables
#   (this increases performance and avoids hard-to-debug behaviour);
# o Look for make include files relative to root of kernel src
MAKEFLAGS += -rR --include-dir=$(CURDIR)

# Avoid funny character set dependencies
unexport LC_ALL
LC_COLLATE=C
LC_NUMERIC=C
export LC_COLLATE LC_NUMERIC

All hope is not lost.  I turns out that you can have make do a “dry run” which will tell you what are the commands that it is going to execute.  Here is part of the output for the Chapter02 example.  Unfortunately, that is some ugly ugly stuff right there.  What am I going to do with that?

The answer is that I am going to do something evil – really evil.  Which you already knew since this is an article about Python and Make you knew coming in that it was going to be evil.  If you notice above there is a line that contains “…. CC [M] …”  That is one of the lines where the compiler is actually being called.  And you might notice that on the command line there are an absolute boatload of “-I” which is the gcc compiler option to add an include path.

The Python Program

What we are going to do here is write a Python program that does this:

  1. Runs make –dry-run
  2. Looks for lines with “CC”
  3. Splits the line up at the spaces
  4. Searches for “-I”s and adds them to a list of include paths
  5. Searches for “-D”s and adds them to a list of #defines
  6. Spits the whole mess out into a json file with the right format (from the Microsoft website)

I completely understand that this program is far far from a robust production worthy program.  But, as it is written in Python, you should not be too surprised.

To start this program off I am going to use several Python libraries

  1. JSON
  2. OS (Operation System so that I can execute make and uname)
  3. RE (Regular expressions)
import json
import os
import re

The next thing to do is declare some global variables.  The first three are Python Sets to hold one copy each of the includes, defines, other options and double dash options.  The Python Set class allows you to add objects to a set that are guaranteed to be unique (if you attempt to add a duplicate it will be dropped)

includePath = set()
defines = set()
otherOptions = set()
doubleDash = set()
outputJson = dict()

The next block of code is a function that:

  1. Takes as an input a line from the makefile output
  2. Splits the line up into tokens by using white space.  The split function take a string and divides it into a list.
  3. Then I iterate over the list (line 27)
  4. I use the Python string slicer syntax – the [] to grab part of the string.  The syntax [:2] means give me the first two characters of the string
  5. I use 4 if statements to look to see if it is a “-I”, “-D”, “–” or “-” in which case I add it to the appropriate global variable.

Obviously this method is deeply hardcoded the output of this version of make on this operating system… but if you are developing Linux Device Drivers you are probably running Linux… so hopefully it is OK.

#
# Function: processLine
#
# Take a line from the make output
# split the line into a list by using whitespace
# search the list for tokens of
# -I (gcc include)
# -D (gcc #define)
# -- (I actually ignore these but I was curious what they all were)
# - (other - options which I keep track of ... but then ignore)

def processLine(lineData):
    linelist = lineData.split()
    for i in linelist:
        if(i[:2] == "-I"):
            if(i[2:2] == '/'):
                includePath.add(i[2:])
            else:
                includePath.add(f"/usr/src/linux-headers-{kernelVersion}/{i[2:]}")
        elif (i[:2] == "-D"):
            defines.add(i[2:])
        elif (i[:2] == "--"):
            doubleDash.add(i)
        elif (i[:1] == '-'):
            otherOptions.add(i)

The next block of code runs two Linux commands (uname and make –dryrun)  and puts the output into a string.  On line 51 I split the make output into a list of strings one per line.

# figure out which version of the kernel we are using
stream = os.popen('uname -r')
kernelVersion = stream.read()
# get rid of the \n from the uname command
kernelVersion = kernelVersion[:-1]

# run make to find #defines and -I includes
stream = os.popen('make --dry-run')
outstring = stream.read()
lines = outstring.split('\n')

In the next block of code I iterate through the makefile output looking for lines that have the “CC” in them.  I try to protect myself by requiring that the CC have white space before and after.  Notice one line 56 that I use a regular expression to look for the “CC”.

for i in lines:
    # look for a line with " CC "... this is a super ghetto method
    val = re.compile(r'\s+CC\s+').search(i)
    if val:
        processLine(i)

The last block of code actually create the JSON and writes it to the output file c_cpp_properties.json.

# Create the JSON 
outputJson["configurations"] = []

configDict = {"name" : "Linux"}
configDict["includePath"] = list(includePath)
configDict["defines"] = list(defines)
configDict["intelliSenseMode"] = "gcc-x64"
configDict["compilerPath"]= "/usr/bin/gcc"
configDict["cStandard"]= "c11"
configDict["cppStandard"] = "c++17"

outputJson["configurations"].append(configDict)
outputJson["version"] = 4

# Convert the Dictonary to a string of JSON
jsonMsg = json.dumps(outputJson)

# Save the JSON to the files
outF = open("c_cpp_properties.json", "w")
outF.write(jsonMsg)
outF.close()

Thats it.  You can then:

  1. Run the program
  2. move the file c_cpp_properties.json in the .vscode directory

And now everything is more better 🙂  When I hover over the “KERN_INFO” I find that it is #define to “6”

I will say that I am not a fan of having the compiler automatically concatenate two strings, but given that this article is written about a Python program who am I to judge?

What could go wrong?

There are quite a few things that could go wrong with this program.

  1. The make output format could change
  2. There could be multiple compiles that have conflicting options
  3. I could spontaneously combust from writing Python programs
  4. The hardcoded cVersion, cStandard, compilerPath, intelliSenseMode could change enough to cause problems

All of these things could be fixed, or at least somewhat mitigated.  But I already spent more time down this rabbit hole that I really wanted.

The Final Program

# This program runs "make --dry-run" then processes the output to create a visual studio code
# c_cpp_properties.json file

import json
import os
import re

includePath = set()
defines = set()
otherOptions = set()
doubleDash = set()
outputJson = dict()

# Take a line from the make output
# split the line into a list by using whitespace
# search the list for tokens of
# -I (gcc include)
# -D (gcc #define)
# -- (I actually ignore these but I was curious what they all were)
# - (other - options which I keep track of ... but then ignore)

def processLine(lineData):
    linelist = lineData.split()
    for i in linelist:
        if(i[:2] == "-I"):
            if(i[2:2] == '/'):
                includePath.add(i[2:])
            else:
                includePath.add(f"/usr/src/linux-headers-{kernelVersion}/{i[2:]}")
        elif (i[:2] == "-D"):
            defines.add(i[2:])
        elif (i[:2] == "--"):
            doubleDash.add(i)
        elif (i[:1] == '-'):
            otherOptions.add(i)

# figure out which version of the kernel we are using
stream = os.popen('uname -r')
kernelVersion = stream.read()
# get rid of the \n from the uname command
kernelVersion = kernelVersion[:-1]

# run make to find #defines and -I includes
stream = os.popen('make --dry-run')
outstring = stream.read()
lines = outstring.split('\n')


for i in lines:
    # look for a line with " CC "... this is a super ghetto method
    val = re.compile(r'\s+CC\s+').search(i)
    if val:
        processLine(i)


# Create the JSON 
outputJson["configurations"] = []

configDict = {"name" : "Linux"}
configDict["includePath"] = list(includePath)
configDict["defines"] = list(defines)
configDict["intelliSenseMode"] = "gcc-x64"
configDict["compilerPath"]= "/usr/bin/gcc"
configDict["cStandard"]= "c11"
configDict["cppStandard"] = "c++17"

outputJson["configurations"].append(configDict)
outputJson["version"] = 4

# Convert the Dictonary to a string of JSON
jsonMsg = json.dumps(outputJson)

# Save the JSON to the files
outF = open("c_cpp_properties.json", "w")
outF.write(jsonMsg)
outF.close()

 

Calibration in a Storm

Summary

A description of an analytic model to adjust pressure sensor depth data to reflect measured data.

Story

About a year ago I repaired my Creek Water Level sensing system.  At that time I installed a new pressure sensor into the system (which had been blown up).  When I did the surgery I did not have data to recalibrate the system.  All along I knew that the system was reading low by about 1 foot or so.  I am  in America, I do Imperial measurement 🙂  But I didn’t really know exactly how much.  I also knew that my original calculate used 0.53ft/psi as the conversion to depth.  This is only true with pure water at about 80 degrees F which meant that it was something else for muddy creek water.

Well on Wednesday last week I had a flood.  So I got the opportunity to collect some real data on the conversion

Old School Measurement

A couple of years ago a friend an I went out with a site level and measure a bunch of marker points, including the base of this treehouse which I know is 12.6 feet over the normal creek level.  When I woke up and saw that the flood was going strong I went out with a long ruler and screwed it into the post holding up the birdhouse.

Here is how it looks close up.

Collect the Data

Unfortunately as the water went higher and higher the only way to collect the data was with a pair of (bad-ass) binoculars.

Over the course of the flood, my children and I would occasionally go out and collect the data.

The next morning I did two things.

  1. Entered the data into a table.
  2. Used mysql to look up the sensor readings at the same time we took the measurement.

Here is the data:

Time Ruler Ruler + BH Sensor Measure
8:15 5 23 13.0 14.6
8:52 7 25 13.4 14.8
9:34 10 28 13.3 15.0
10:17 12 30 13.5 15.2
10:46 13 31 13.6 15.3
12:54 18 36 14.0 15.7
13:43 21 39 14.2 16.0
15:00 26 44 14.7 16.4
16:18 30 48 15.0 16.7
17:50 33 51 15.2 17.0
7:55 15 33 13.8 15.5

Analyze the Data

The next step was to analyze the data.  So, I created an x-y plot.  Notice the red datapoint almost certainly was read in error.  The dotted line is an excel created least squares fit of the data.

When I remove the red dot I get a correlation coefficient of 0.9951 … that is money in my business.

Now when I create column the new model you can see that all of the datapoints are within 1%.

Time Ruler Ruler + BH Sensor Measure Model Error RMS
8:15 5 23 13.0 14.6 14.7 0.8%             0.0
8:52 7 25 13.4 15.1
9:34 10 28 13.3 15.0 15.0 -0.3%             0.0
10:17 12 30 13.5 15.2 15.2 -0.2%             0.0
10:46 13 31 13.6 15.3 15.3 -0.1%             0.0
12:54 18 36 14.0 15.7 15.7 -0.2%             0.0
13:43 21 39 14.2 16.0 15.9 -0.5%             0.0
15:00 26 44 14.7 16.4 16.4 0.36%             0.0
16:18 30 48 15.0 16.7 16.7 0.10%             0.0
17:50 33 51 15.2 17.0 17.0 0.02%             0.0
7:55 15 33 13.8 15.5 15.5 0.1%             0.0
0.011

Here is a plot of the error:

Fix the Firmware

The next step is to update the firmware on sensor system.  The comment on line 56 of the code “USC correction model” means that I talked with the guy in charge of transistor device modeling at Infineon/Cypress.  He suggested some improvements from my original analysis.

dp.pressureCounts = adc_GetResult16(PRESSURE_CHAN);
            // 408 is the baseline 0 with no pressure = 51.1ohm * 4ma * 2mv/count
            // 3.906311 is the conversion to ft
            // Whole range in counts = (20mA - 4mA)* 51.1 ohm * 2 mv/count = 1635.2 counts
            // Range in Feet = 15PSI / 0.42PSI/Ft = 34.88 Ft
            // Count/Ft = 1635.2 / 34.88 = 46.8807 Counts/Ft 
            
            float depth =( ((float)dp.pressureCounts)-408)/46.8807;
            
            // Apply the USC correction model
            depth = 1.0106 * depth + 1.5589;
            
            dp.depth = ( depth + 7*previousDepth ) / 8.0;  // IIR Filter

The last thing to do is fix all of the old data in my database.  So I use mysql to update all of the datapoint with the adjusted values since I installed the new sensor.

update creekdata set depth=depth*1.016 + 1.5589 where id>=1749037 AND id<= 1981538;

This morning while I was doing the updates the creek started flooding again.  Here is the plot where you can see the offset being applied.

And with the offset applied, things are “more better” as my mom would say.

 

PSoC 6 TCPWM Pulse Width Measurement

Summary

This article walks you through a PSoC 6 SDK example of measuring a pulse width using the TCPWM block’s counter function.

The Story

Recently I have been working with a Cypress guy and frequent collaborator Hassane.  He is working on a project that includes some kind of sonic distance sensor (I am actually not sure which one).  In order to “read” the sensor you need to be able to measure the width of a pulse which is easy enough to do using the TCPWM.  I originally wanted to use the Cypress HAL to setup the PSoC 6 to do this function, but it is not yet enabled.  So, I need to use PDL and the PSoC 6 Configurator.

Configure The TCPWM

Start with a new PSoC 6 project.  In my case I am using the CY8CKIT-062-WiFi-BT development kit because I had one where I had already soldered a wire onto the user switch.  My plan is to create a “pulse” with the user switch, starting from the press, ending with the user letting go.  This will be an active low pulse.  I will configure the TCPWM to have start, reload setup as Falling edges.  In other words the counter will reset and start counting when the button is pressed.  Then I will setup Stop and Reload on Rising edges, in other words the timer will STOP and give an interrupt on the rising edge.  All of these pins are connected to P9[0] on the PSoC.

You might ask yourself, why P9[0]?  That development kit has a switch connected to P0[4] seems like you should have connected the TCPWM to that pin.  That was done because the TCPWM is only attached to a limited set of signals.  Here is a screenshot from the configurator.

I want this setup to be interrupt based, specifically I want an interrupt when the Capture signal is active.  It is also interesting when the counter rolls over (overflow).  So I setup the interrupt source to be “Overflow & Underflow or Compare & Capture”.

The last thing you need is a clock.   you now need a clock.  To do this clock on the Clock Signal.  For this example I decided to use a 16-bit clock.

But what frequency is the input clock?  To set this go to the “Peripheral-Clocks”  The click on the divider you choose from above.  Then pick a divide value.  I choose 10000 which will give me 100MHz/10K=10KHz.

But where did the 100MHz come from?  Click on the “System” and look at the clock diagram.  All of the “Peripheral Clocks” use the signal “CLK_PERI” as their source.  This is attached to CLK_HF0 (which is 100Mhz) in this case.

The Program

Once is everything is configured you need to write a bit of code.

9-14: First I create an ISR which will set a flag and clear the interrupt source in the TCPWM

Then I write a main function

20: Turn on the printf functionality using the cy_retarget_io library

25: Initialize the counter using the structure created by the configurator.

26: Then I enable the counter to run.  Be careful with this function because counter numbers are NOT the same thing as counter masks.  Counter 3 is counter mask 1<<3=8

28-30: Turn on the interrupts and register my ISR called counterDone

34: In the loop I wait around until the flag gets set by the ISR

36: Read the value from the counter and convert it to seconds

37: Print the value

38: Reset the flag

And do it all over again

#include "cy_pdl.h"
#include "cyhal.h"
#include "cybsp.h"
#include "cy_retarget_io.h"
#include <stdio.h>

volatile int flag=0;

void counterDone()
{
    uint32 intCause = Cy_TCPWM_GetInterruptStatusMasked(MYC_HW,MYC_NUM);
    flag = 1;
    Cy_TCPWM_ClearInterrupt(MYC_HW,MYC_NUM,intCause);
}

int main(void)
{
    cybsp_init() ;
    
    cy_retarget_io_init(CYBSP_DEBUG_UART_TX, CYBSP_DEBUG_UART_RX, CY_RETARGET_IO_BAUDRATE);
    __enable_irq();

    printf("Started\n");

    Cy_TCPWM_Counter_Init(MYC_HW,MYC_NUM,&MYC_config);
    Cy_TCPWM_Enable_Multiple(MYC_HW,MYC_MASK);

    cy_stc_sysint_t mycounter= {.intrSrc = MYC_IRQ, .intrPriority=7};
    Cy_SysInt_Init(&mycounter,counterDone);
    NVIC_EnableIRQ(MYC_IRQ);

    for(;;)
    {
        if(flag == 1)
        {
            float val = (float)Cy_TCPWM_Counter_GetCounter(MYC_HW,MYC_NUM) / 10000.0;
            printf("%.2fs\n",val);
            flag = 0;
        }
    }
}

Test

For an earlier article I had to solder a wire onto the switch.  This let me attach it to an oscilliscope.  It turns out that was an easy way to attach the switch to P9[0]

When I run the project I get a bunch of different pulse.  Looks good.