View source for CRC Checksum Generation with ‘SRecord’ Tools for GNU and Eclipse

One of the things missing for Embedded in the GNU linker is that it cannot generate a CRC checksum. Luckily, there is the solution of using SRecord:

[[File:Srecord-1-64-web-page.png]]

'''Outline'''

Because the GNU (e.g. GNU ARM Embedded/launchpad) toolchain does not include a CRC checksum calculation function, I’m showing how the SRecord utility can be used for this. The SRecord tool is an open source project on SourceForge (http://srecord.sourceforge.net/). It is a command line utility which runs on many platforms. I’m using it in this post with Eclipse (e.g. Freescale Kinetis Design Studio, or any other Eclipse based toolchain using the GNU ARM Embedded (Liviu, http://gnuarmeclipse.livius.net/) with GNU for ARM (GNU for ARM Embedded (Launchpad): https://launchpad.net/gcc-arm-embedded). It goes through the steps to create a checksum, add it to the binary image and checking that checksum in the application.

''💡 As the name ‘SRecord’ suggests it deals with S-Records (or S19) files. This is just the default format. SRecord can read and generate pretty much any file format which is used for programming memory devices or microcontroller.''


'''Installation'''

Go to http://srecord.sourceforge.net/ and download the binaries of your choice from http://srecord.sourceforge.net/download.html. Of course you can as well download the sources and build it yourself. That site hosts as well a lot of good documentation, but if you are a (mostly) Windows user as I am, then be prepared for some Windows user bashing ;-).

SRecord comes with three utilities:

1. srec_info: used to retrieve basic information about the file. It reports things like start address.

2. srec_cmp: used to compare two files. This utility only tells you if two files are (memory-wise) different or not, but not more.

3. srec_cat: This tool is used to extract/add/create/merge/etc files.


All the three tools are command line tools and have extensive support for options. So they easily can be used with make files, scripts or from IDEs as Eclipse. See http://srecord.sourceforge.net/man/index.html for the online manual.

''💡 The cool part is that they support ‘input generators’ and ‘filters’, see http://srecord.sourceforge.net/man/man1/srec_input.html''


'''Generating S-Record Files'''

Usually the linker main output file is an ELF/Dwarf file which has both code and debug information. The ELF/Dwarf file is used for debugging. All toolchains I’m aware of are able to generate more output files beside of the ELF/Dwarf: S-Record (S19), Intel Hex, etc files. For example in Kinetis Design Studio use the ‘Create flash image’ option in the project settings and press ‘Apply’:

[[File:Create-flash-image.png]]


Then you can select the ‘Motorola S-record’ option in the project settings:

[[File:Motorola-s-record-option.png]]


The generated S19 file can be found in the Debug output folder:

[[File:S19-file.png]]


'''srec_info'''

srec_info gives basic information about the file:

 srec_info Debug\FRDM-KL25Z_CRC.srec

gives

 Format: Motorola S-Record
 Header: "FRDM-KL25Z_CRC.srec"
 Execution Start Address: 000007CD
 Data:   0000 - 00BF
         0400 - 0C3B

See http://srecord.sourceforge.net/man/man1/srec_info.html for further information


'''srec_cmp'''

srec_cmp is a program which can compare two files. Unlike a normal diff, it compares two ‘memory’ files. For example

 srec_cmp app1.srec app2.srec

gives

 srec_cmp: files "app1.srec" and "app2.srec" differ

See http://srecord.sourceforge.net/man/man1/srec_cmp.html for further information.


'''srec_cat'''

srec_cat is the ‘main’ program of the suite. As the name indicates it can concatenate multiples files. But it can do much more:

-Converting files 

-Inserting or removing data

-Joining/splitting files

-Moving data

-Fill in patterns or fill the blanks

-Creating data

-Changing data

-and of course creating multiple kinds of checksums 🙂

There are many good examples how to use it here: http://srecord.sourceforge.net/man/man1/srec_examples.html


'''Crop and Generating a Hex Dump'''

One thing I’m using often is to do a memory dump of my s-record. I can do this with the -hex-dump option:

 srec_cat -crop -Output - -hex-dump

''💡 After the -Output option there is usually a file name. Using ‘-‘ as file name will write the output to the console.''

For example

 srec_cat FRDM-KL25Z_CRC.srec -crop 0x500 0x530 -Output - -hex-dump

produdes

 00000500: BD 46 80 BD B0 B5 82 B0 00 AF 78 60 39 60 10 4C  #=F.=05.0./x`9`.L
 00000510: 00 25 15 E0 23 1C 1B 02 9A B2 23 0A 9B B2 19 1C  #.%.`#....2#..2..
 00000520: 7B 68 58 1C 78 60 1B 78 5B B2 59 40 FF 23 19 40  #{hX.x`.x[2Y@.#.@

I use that approach to quickly inspect or dump content of my image (.elf/.s19) file. Additionally, this allows me to inspect the memory of the target and compare it with what I have in my file.

''💡 The -crop command crops (or cuts) everything out of the data except the range specified (the end address not included).''


'''Filling Memory'''


Typically a data file as the Motorola S19/SRecord only describes the bytes to be programmed, but not the ‘holes’ or gaps in the memory map. If building a CRC over a memory area with gaps, I need to define it first. For this I can use the  -fill command:
 -fill  [ ]

To fill multiple areas, e.g. to fill 0x100-0x1FF and 0x300-0x3FF with 0xFF, I could use

 -fill 0xFF 0x100 0x200 -fill 0xFF 0x300 0x400

Or I could use a list of <start>…<end> addresses:
 -fill 0xFF 0x100 0x200 0x300 0x400

For the microcontroller on the FRDM-KL25Z board (a KL25Z128) I have the following memory map:

 MEMORY {
  m_interrupts (RX) : ORIGIN = 0x00000000, LENGTH = 0x000000C0
  m_text      (RX) : ORIGIN = 0x00000410, LENGTH = 0x0001FBF0
  m_data      (RW) : ORIGIN = 0x1FFFF000, LENGTH = 0x00004000
  m_cfmprotrom  (RX) : ORIGIN = 0x00000400, LENGTH = 0x00000010
 }

To fill the interrupt table (m_interrupts) and the code (m_text) with 0xFF and dump it, I can use

 -fill 0xFF 0x0000 0x00C0 0x0410 0x20000


'''Generating CRC'''

srec_cat supports many checksums. The challenge for me was to match the srec_cat way of generating the checksum with the right algorithm and polynom. An online CRC calculation utility on http://www.lammertbies.nl/comm/info/crc-calculation.html helped me to identify the matching CRC polynom and algorithm. In my applications I’m using the Big Endian CCITT CRC16. To generate it, use the following command:

 -CRC16_Big_Endian 0x1FFFE -CCITT

-'''CRC16_Big_Endian or -crc16-b-e''' is used to store the CRC is stored in Big Endian format.


'''CRC16 Source Files'''

http://www.menie.org/georges/embedded/crc16.html is a great source for a CRC16 calculation function. Another even better one is on http://www.sunshine2k.de/coding/javascript/crc/crc_js.html. I have updated the source to match the 0x1D0F starting point which is used for -CRC16_Big_Endian. I changed the implementation to use a defined starting point: The interface file:

 /*
 * Copyright 2001-2010 Georges Menie (www.menie.org)
 * All rights reserved.
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 *     * Redistributions of source code must retain the above copyright
 *       notice, this list of conditions and the following disclaimer.
 *     * Redistributions in binary form must reproduce the above copyright
 *       notice, this list of conditions and the following disclaimer in the
 *       documentation and/or other materials provided with the distribution.
 *     * Neither the name of the University of California, Berkeley nor the
 *       names of its contributors may be used to endorse or promote products
 *       derived from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY
 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE REGENTS AND CONTRIBUTORS BE LIABLE FOR ANY
 * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */
 
 #ifndef _CRC16_H_
 #define _CRC16_H_
 
 #define CRC16_START_VAL  0x1D0F /* start value which is used by SRecord tool for -CRC16_Big_Endian */
 
 unsigned short crc16_ccitt(const void *buf, int len, unsigned short start);
 
 #endif /* _CRC16_H_ */


The implementation file:

 /*
 * Copyright 2001-2010 Georges Menie (www.menie.org)
 * All rights reserved.
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 *     * Redistributions of source code must retain the above copyright
 *       notice, this list of conditions and the following disclaimer.
 *     * Redistributions in binary form must reproduce the above copyright
 *       notice, this list of conditions and the following disclaimer in the
 *       documentation and/or other materials provided with the distribution.
 *     * Neither the name of the University of California, Berkeley nor the
 *       names of its contributors may be used to endorse or promote products
 *       derived from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND ANY
 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL THE REGENTS AND CONTRIBUTORS BE LIABLE FOR ANY
 * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */
 
 #include "crc16.h"
 
 /* CRC16 implementation according to CCITT standards */
 
 static const unsigned short crc16tab[256]= {
    0x0000,0x1021,0x2042,0x3063,0x4084,0x50a5,0x60c6,0x70e7,
    0x8108,0x9129,0xa14a,0xb16b,0xc18c,0xd1ad,0xe1ce,0xf1ef,
    0x1231,0x0210,0x3273,0x2252,0x52b5,0x4294,0x72f7,0x62d6,
    0x9339,0x8318,0xb37b,0xa35a,0xd3bd,0xc39c,0xf3ff,0xe3de,
    0x2462,0x3443,0x0420,0x1401,0x64e6,0x74c7,0x44a4,0x5485,
    0xa56a,0xb54b,0x8528,0x9509,0xe5ee,0xf5cf,0xc5ac,0xd58d,
    0x3653,0x2672,0x1611,0x0630,0x76d7,0x66f6,0x5695,0x46b4,
    0xb75b,0xa77a,0x9719,0x8738,0xf7df,0xe7fe,0xd79d,0xc7bc,
    0x48c4,0x58e5,0x6886,0x78a7,0x0840,0x1861,0x2802,0x3823,
    0xc9cc,0xd9ed,0xe98e,0xf9af,0x8948,0x9969,0xa90a,0xb92b,
    0x5af5,0x4ad4,0x7ab7,0x6a96,0x1a71,0x0a50,0x3a33,0x2a12,
    0xdbfd,0xcbdc,0xfbbf,0xeb9e,0x9b79,0x8b58,0xbb3b,0xab1a,
    0x6ca6,0x7c87,0x4ce4,0x5cc5,0x2c22,0x3c03,0x0c60,0x1c41,
    0xedae,0xfd8f,0xcdec,0xddcd,0xad2a,0xbd0b,0x8d68,0x9d49,
    0x7e97,0x6eb6,0x5ed5,0x4ef4,0x3e13,0x2e32,0x1e51,0x0e70,
    0xff9f,0xefbe,0xdfdd,0xcffc,0xbf1b,0xaf3a,0x9f59,0x8f78,
    0x9188,0x81a9,0xb1ca,0xa1eb,0xd10c,0xc12d,0xf14e,0xe16f,
    0x1080,0x00a1,0x30c2,0x20e3,0x5004,0x4025,0x7046,0x6067,
    0x83b9,0x9398,0xa3fb,0xb3da,0xc33d,0xd31c,0xe37f,0xf35e,
    0x02b1,0x1290,0x22f3,0x32d2,0x4235,0x5214,0x6277,0x7256,
    0xb5ea,0xa5cb,0x95a8,0x8589,0xf56e,0xe54f,0xd52c,0xc50d,
    0x34e2,0x24c3,0x14a0,0x0481,0x7466,0x6447,0x5424,0x4405,
    0xa7db,0xb7fa,0x8799,0x97b8,0xe75f,0xf77e,0xc71d,0xd73c,
    0x26d3,0x36f2,0x0691,0x16b0,0x6657,0x7676,0x4615,0x5634,
    0xd94c,0xc96d,0xf90e,0xe92f,0x99c8,0x89e9,0xb98a,0xa9ab,
    0x5844,0x4865,0x7806,0x6827,0x18c0,0x08e1,0x3882,0x28a3,
    0xcb7d,0xdb5c,0xeb3f,0xfb1e,0x8bf9,0x9bd8,0xabbb,0xbb9a,
    0x4a75,0x5a54,0x6a37,0x7a16,0x0af1,0x1ad0,0x2ab3,0x3a92,
    0xfd2e,0xed0f,0xdd6c,0xcd4d,0xbdaa,0xad8b,0x9de8,0x8dc9,
    0x7c26,0x6c07,0x5c64,0x4c45,0x3ca2,0x2c83,0x1ce0,0x0cc1,
    0xef1f,0xff3e,0xcf5d,0xdf7c,0xaf9b,0xbfba,0x8fd9,0x9ff8,
    0x6e17,0x7e36,0x4e55,0x5e74,0x2e93,0x3eb2,0x0ed1,0x1ef0
 };
 
 unsigned short crc16_ccitt(const void *buf, int len, unsigned short start)
 {
    register int counter;
    register unsigned short crc = start;
 
    for( counter = 0; counter < len; counter++) {
        crc = (crc<<8) ^ crc16tab[((crc>>8) ^ *(char *)buf++)&0x00FF];
    }
    return crc;
 }


'''Checking CRC in the Application'''

Below is a piece of code in the application which checks the CRC:

 /*
 * Application.c
 *
 *  Created on: 16.04.2015
 *      Author: Erich Styger
 */
 
 #include "Application.h"
 #include "crc16.h"
 
 #define CRC_RANGE_START_ADDR    0x00410 /* start addr */
 #define CRC_RANGE_END_ADDR      0x1FFFE /* end addr, this one will not be counted */
 #define CRC_VALUE_ADDR          0x1FFFE /* address of CRC (16bits) */
 
 void APP_CheckCRC(void) {
  unsigned short crc;
 
  crc = crc16_ccitt((void*)CRC_RANGE_START_ADDR, CRC_RANGE_END_ADDR-CRC_RANGE_START_ADDR, CRC16_START_VAL);
  if (crc!=*((unsigned short*)CRC_VALUE_ADDR)) {
    for(;;) {} /* error! CRC does not match! */
  }
 }

The above code assumes that the 16bit CRC is stored at address 0x1FFFE. The application checks the CRC for the code space from address 0x410 up to 0x1FFFD (which does *not* include the CRC itself which is at 0x1FFFE-0x1FFFF).


'''Using Command Files'''

Instead passing everything on the command line, I can call the SRecord tool suite programs with the options in an external file. The syntax is

 <SrecordProgram> @filename

For example

 srec_cat @crc_cmd.txt


'''Dumping the CRC for a memory range'''

Using the ‘@’ syntax, I can execute a script which calculates the CRC and dumps it for me so I can manually check it. For this I use the following content of a command file:

 # srec_cat command file to dump the CRC for a code area
 # Usage: srec_cat @filename
 FRDM-KL25Z_CRC.srec          # input file
 -fill 0xFF 0x0410 0x20000          # fill code area with 0xff
 -crop 0x0410 0x1FFFE               # just keep code area for CRC calculation below (CRC will be at 0x1FFFE..0x1FFFF)
 -CRC16_Big_Endian 0x1FFFE -CCITT   # calculate big endian CCITT CRC16 at given address.
 -crop 0x1FFFE 0x20000              # keep the CRC itself
 -Output                            # produce output
 -                                  # '-' is special 'file': use console output
 -hex-dump                          # dump in hex format

That way I can have things commented (comments start with ‘#’) and keep things readable. The above program calculates the CRC over a given range, stores the value at the artificial address 0x20000 (outside of the code area) and dumps the 16bit value on the console:

[[File:Executing-script-to-dump-crc-value2.png]]


'''Incorporating the CRC Value into the Application'''

Knowing the CRC, I need to incorporate the CRC value itself into my application. One way would be to do this in the GNU linker script itself (see “FILLing unused Memory with the GNU Linker“). However, that would be a manual process:

1. Determine the CRC value with SRecord and dump it

2. Enter the CRC value into the linker script


Another approach would be:

1. Generate the CRC with SRecord and produce the S19 file which only has the CRC in it (CRC S19 file)

2. Merge the application S19 file with the CRC S19 file


This approach works very well, as srec_cat (as the name indicates) is excellent to concatenate files :-). To produce the CRC S19 file I can use the following command file:

 FRDM-KL25Z_CRC.srec                # input file
 -fill 0xFF 0x0410 0x20000          # fill code area with 0xff
 -crop 0x0410 0x1FFFE               # just keep code area for CRC calculation below (CRC will be at 0x1FFFE..0x1FFFF)
 -CRC16_Big_Endian 0x1FFFE -CCITT   # calculate big endian CCITT CRC16 at given address.
 -crop 0x1FFFE 0x20000              # keep the CRC itself
 -Output                            # produce output
 FRDM-KL25Z_CRC.srec                # S19 with CRC only


With this, I have only the CRC in FRDM-KL25Z_CRC.srec.


Then I need to fill the unused areas in the application file. Here again, I can generate a ‘filled’ file:

 srec_cat FRDM-KL25Z_CRC.srec -fill 0xFF 0x0410 0x1FFFE -Output FRDM-KL25Z_CRC_Filled.srec


Finally, concatenate the two files with

 srec_cat FRDM-KL25Z_CRC_Filled.srec FRDM-KL25Z_CRC.srec -Output Debug\FRDM-KL25Z_CRC_Added.srec


However, this involves using temporary files which is not ideal. A better approach is to do everything in one step, with a single command file like this:

 # srec_cat command file to add the CRC and produce application file to be flashed
 # Usage: srec_cat @filename
 #first: create CRC checksum
 FRDM-KL25Z_CRC.srec                # input file
 -fill 0xFF 0x0410 0x20000          # fill code area with 0xff
 -crop 0x0410 0x1FFFE               # just keep code area for CRC calculation below (CRC will be at 0x1FFFE..0x1FFFF)
 -CRC16_Big_Endian 0x1FFFE -CCITT   # calculate big endian CCITT CRC16 at given address.
 -crop 0x1FFFE 0x20000              # keep the CRC itself
 #second: add application file
 FRDM-KL25Z_CRC.srec                # input file
 -fill 0xFF 0x0410 0x1FFFE          # fill code area with 0xff
 #finally, produce the output file
 -Output                            # produce output
 FRDM-KL25Z_CRC_Added.srec

To me, this is really a cool thing of the SRecord tool: the ability to line up files and content and then merge it 🙂

And as expected: the application image file has the CRC added:


[[File:Crc-added-to-the-application-image-file.png]]


'''Adding CRC as Post-Build Step'''


Now I have the ability to add the CRC to my application file. It would be great if this could be part of my build process? Using normal make files, I simply would call srec_cat after linking. The same thing can be done with Eclipse as ‘Post-Build-Step’. For example to dump the CRC value, I can have this in the project settings:


[[File:Post-build-step-to-dump-crc.png]]


''💡 Keep in mind that the ‘current directory’ for the build process is the output folder, usually the ‘Debug’ folder.''


And it will dump the CRC at the end of the build process:


[[File:Srec cat-as-post-build-step-in-the-eclipse-console-view.png]]


The same way I can add the CRC to the application:


[[File:Adding-crc-to-application.png]]


However, this will fail for a clean build:


[[File:Error-while-adding-crc.png]]


The reason is that the post build step is executed *before* the S19 file is generated. The solution is to add the generation of the S19 file to the post build step too. It is possible to execute multiple post-build steps (see “Executing Multiple Commands as Post-Build Steps in Eclipse“), just be aware that the separator on Windows is ‘@’ and it is ‘;’ on Linux:

''💡 One strange thing I noticed: as soon as I had multiple commands, I had to use ‘/’ instead of ‘\’ on Windows.''


[[File:Generating-s19-and-crc.png]]


With this, everything works as expected:

[[File:Console-output-generating-s19-file-and-adding-crc.png]]

'''Summary'''

Generating a CRC is not possible directly with the GNU linker. But this is not a problem, as there is an even more powerful way with the ‘SRecord’ utilities: with the SRecord utilities I can almost any manipulation of the output files I need, plus best of all: it is open source too 🙂

I have put my project used in this post on GitHub here: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/KDS/FRDM-KL25Z/FRDM-KL25Z_CRC

'''LINKS'''

-‘SRecord’ home page: http://srecord.sourceforge.net/

-‘SRecord’ examples: http://srecord.sourceforge.net/man/man1/srec_examples.html

-Online CRC calculation: http://www.lammertbies.nl/comm/info/crc-calculation.html

-GNU ARM Eclipse Plugins: http://gnuarmeclipse.livius.net/

-GNU for ARM Embedded (Launchpad): https://launchpad.net/gcc-arm-embedded

-Generating S19 Files with CodeWarrior: https://mcuoneclipse.com/2012/09/13/s-record-generation-with-gcc-for-armkinetis/

-Generating S19 Files with Kinetis Design Studio: https://mcuoneclipse.com/2014/04/20/binary-files-for-the-mbed-bootloader-with-eclipse-and-gnu-arm-eclipse-plugins/

-CRC16 CCITT Example source code: http://www.menie.org/georges/embedded/crc16.html

-Post-Build-Steps in Eclipse: Executing Multiple Commands as Post-Build Steps in Eclipse