korena's blog

15. Bootloader Implementation - part 5

In the previous post, we looked into the initialization process we have to go through to get our DM9000A chip to wake up and start receiving traffic, we haven't seen this though, so we'll start actually compiling our code, and looking at the terminal, to see if we can go through initialization without errors, then we'll move to develop our dm9000_rx function, which will be responsible for receiving data from the internal dm9000A chip rx buffer, and passing it for further analysis to the appropriate protocol handler.

Initialization test

To kick start our initialization process, we have to call the function dm9000_initialize(), but before we can do that, we have to build a couple of blocks so that we can keep our bootloader.c code far enough from the actual network stack implementation, so we'll create a bl2/src/net.c, loosely following u-boot's implementation where we can. This file or 'layer' will mainly be responsible for managing and dispatching protocol level handling, it will also be our bootloader's entry point, and exit point to the networking stack. The first thing we'll put in this file is a function that (eventually) calls our driver's dm9000_initialize function:

void net_init(void)
{
        static int first_call = 1;
        if (first_call) {
                /*
                 *      Setup packet buffers, aligned correctly.
                 */
                int i;

                net_ip = string_to_ip("10.0.0.2");       
                net_server_ip = string_to_ip("10.0.0.20");
                net_tx_packet = &net_pkt_buf[0] + (PKTALIGN - 1);
                net_tx_packet -= (unsigned long)net_tx_packet % PKTALIGN;
                for (i = 0; i < PKTBUFSRX; i++) {
                        net_rx_packets[i] = net_tx_packet +
                                (i + 1) * PKTSIZE_ALIGN;
                }
                print_format("initializing ARP ...\n\r");
                arp_init();
                print_format("Done initializing ARP ...\n\r");
                /* Only need to setup buffer pointers once. */
                first_call = 0;
        }

        /*initialize ethernet device*/
        if(dm9000_initialize() == 0){
                /* device is initialized! register generic listener for requests...*/
                print_format("dm9000_initialization complete.\n\r");
                eth_current->state = ETH_STATE_ACTIVE;
                return;
        }else{
                print_format("net_init failed at ethernet device initialization, aborting.\n\r");
                return; 
        }

}

We have some basic initialization at work here, ignore the calls to set IP addresses and initializing ARP, they're not of any consequence to what we're trying to achieve here, and we'll get back to them in a later post (in fact, they're probably nonfunctional). For now, we're focusing on line 26 ,which calls our Ethernet chip initialization function, we see that initialization is successful only if the function returns zero, internally, the initialization function calls a routine that sets a global structure called eth_current, which is of type struct eth_device :

int dm9000_initialize(void)
{
        int i;
        int regStat;
        struct eth_device *dev = &(dm9000_info.netdev);
        /*Fill MAC address ...*/        
        for(i = 0;i<6;i++)
                dev->enetaddr[i] = net_ethaddr[i];
        if (dm9000_init(dev) == 0){
                dev->init = dm9000_init;
                dev->halt = dm9000_halt;
                dev->send = dm9000_send;
                dev->recv = dm9000_rx;
                sprintf((char*)dev->name, "dm9000");

                regStat = eth_register(dev);

                switch(regStat){
                        case 0:{
                                       print_format("eth_current successfully registered\n\r");
                                       return 0;
                               }
                        case 1:{ 
                                       print_format("dev was null, eth_current not registered\n\r");
                                       return 1;
                               }
                        case 2:{ 
                                       print_format("dev name too long, eth_current not registered\n\r");
                                       return 2;
                               }
                        case 3:{ 
                                       print_format("eth_current is null, eth_current not registered\n\r");// should not happen
                                       return 3;
                               }
                        default:
                               return 4;

                }
        }else{
                print_format("Failed to initialize dm9000\n\r");
                return -1;
        }

}

In line 16, a function eth_register(dev) is called, and passed the eth_device structure that is initialized at the driver level. Since we have only one Ethernet device, we could have dropped the use of the global eth_current all together, but it keeps us closer to u-boot's implementation, and it seemed like a good idea at the time, I may revisit this at some point (read: will never happen.). Moving on, the eth_register function is implemented in another file I created, which is bl2/src/eth.c, this file will sit between the driver (bl2/src/dm9000.c) and the network protocol manager (bl2/src/net.c), it's primary function is to abstract the driver specific functions, so if we are to change the Ethernet chip used, and consequently the driver, we would only have to deliver a valid struct eth_device from the driver, and we'd be good to go, without changing any of the other files (theoretically, this is what you would aim for, but we're not paying too much attention to that now, so I have a feeling we're going to break it real soon). The eth_register function looks like:

int eth_register(struct eth_device *dev)
{
	if(dev == NULL)
		return 1;

	if(strlen((const char*)dev->name) > sizeof(dev->name)){
		print_format("error registering ethernet device, name too long\n\r");
		print_format("length of name:%d\n\r",sizeof(dev->name));
	return 2;
	}
	
	dev->state = ETH_STATE_INIT;
	eth_current = dev;
	if(eth_current == NULL)
		return 3;

	return 0;
}

Lines 13 through 15 make no sense, but frustration gets you there, don't judge! The function defines the state of the eth_device, which is used later to decide stuff in the net.c main network loop.

As stated, file net.c will be our entry point to the whole mess below, so we have to expose a function for our bootloader (specifically, bootloader.c file) to call, this function should initialize the network stack, and only return when it has achieved whatever it is that our bootloader asked of it, this means that this function should be able to accept commands (read arguments), it should also be able to detect whether is should initialize the driver, or simply use the already initialized driver to deliver the bootloader's request, so we cannot elect net_init to be our bootloader's entry point (at least not the way it's implemented now). what we're going to do is implement a

while(net_stack_hasnt_delivered_whats_required_from_it){lock, and keep trying}

This is the first serious divergence from u-boot's way, while u-boot uses a state machine, we will be using a dumb loop with some conditions to break out. So I added the following function to bl2/src/net.c:

/*
 * We want this to handle ARP,TFTP and PING, nothing more!
 * */
int net_loop(enum proto_t protocol)
{
        int ret = -EINVAL;

        net_restarted = 0;
        net_dev_exists = 0;
        net_try_count = 1;
        print_format("--- net_loop Entry\n\r");

        print_format("eth_start, calling net_init\n\r");
        net_init();
restart:
        net_set_state(NETLOOP_CONTINUE);

        /*
         *      Start the ball rolling with the given start function.
         */
        print_format("--- net_loop Init\n\r");

        switch (net_check_prereq(protocol)) {
                case 1:
                        /* network not configured */
                        print_format("-ERR network not configured, halting device ...\n\r");
                        eth_halt();
                        return -ENODEV;

                case 2:
                        /* network device not configured */
                        print_format("-ERR network device not configured ...\n\r");
                        break;

                case 0:
                        net_dev_exists = 1;
                        net_boot_file_size = 0;
                        print_format("acting on chosen protocol ...\n\r");
                        switch (protocol) {
                                case TFTPGET:
                                        /* always use ARP to get server ethernet address */
                //                      tftp_start(protocol);
                                        break;
                                case BOOTP:
                                        break;
                                case PING:
                                        break;
                                case LINKLOCAL:
                                        break;
                                default:
                                        break;
                        }

                        break;
        }

        /*
         *      Main packet reception loop.  Loop receiving packets until
         *      someone sets `net_state' to a state that terminates.
         */
        print_format("looping and polling ethernet recv ...\n\r");
        int ethStatus ;
        for (;;) {
                /*
                 *      Check the ethernet for a new packet.  The ethernet
                 *      receive routine will process it.
                 *      Most drivers return the most recent packet size, but not
                 *      errors that may have happened.
                 */
                ethStatus = eth_rx();
                if (net_state == NETLOOP_FAIL)
                        print_format("some failure was encounterd, good luck debugging !!\n\r");
                switch (net_state) {
                        case NETLOOP_RESTART: /*I should never happen ..*/
                                net_restarted = 1;
                                goto restart;

                        case NETLOOP_SUCCESS:
                                print_format("--- net_loop Success!\n\r");
                                goto done;

                        case NETLOOP_FAIL:
                                print_format("--- net_loop Fail!\n");
                                goto done;

                        case NETLOOP_CONTINUE:
                                continue;
                }
        }

done:
        return ret;
}

This will be the entry point for our bootloader.c file, since we have no interactive user interface (we should), we can simply call things sequentially, and just reset the processor when we see something we don't like, this can be improved later.

Now, for testing the initialization functionality, we only need to pay attention to line 14, which is the call to initialize the Ethernet chip, so let's get things going, in our bootloader.c file:

start_linux(void)
{
        debug_print("Setting up timer ...\n\r");
        init_timer();   
        debug_print("About to initialize ethernet networking ...\n\r"); 
        net_loop(ARP);
while(1){
        debug_print("delaying for 1 second ...\n\r");
        udelay(1000000);
        debug_print("done!\n\r");
}

    return 0;
}

We trimmed down our start_linux function to test the DM9000A Ethernet chip initialization, and called net_loop(irrelevant argument). Our test has to pass two things:

  • We should see all the good "successful" messages printed in the console
  • We should be able to establish a connection, a low level confirmation of established connection has to be confirmed by another NIC module at the other end of the wire.

This last point doe's not mean that our board has to be recognized completely, meaning it doesn't have to respond to ARP requests or anything like that, but it has to be able to establish connection at the PHY level, so it should be capable of completing auto negotiation of connection speed, which only requires proper initialization, but nothing more. So let's get to it.

/home/korena/Development/Workspaces/car-dashboard/net_stack$ make clean && make && make fuse

we have a ready to go MMC card, slot it in, connect a crossover cable between development board and development machine, then:

/home/korena/Development/Workspaces/car-dashboard/net_stack/host_scripts$ kermit kermrc
C-Kermit 9.0.302 OPEN SOURCE:, 20 Aug 2011, for Linux+SSL+KRB5 (64-bit)
 Copyright (C) 1985, 2011,
  Trustees of Columbia University in the City of New York.
Type ? or HELP for help.
(/home/korena/Development/Workspaces/car-dashboard/net_stack/host_scripts/) C-Kermit>connect
Connecting to /dev/ttyUSB0, speed 115200
 Escape character: Ctrl-\ (ASCII 28, FS): enabled
Type the escape character followed by C to get back,
or followed by ? to see other options.
----------------------------------------------------

Powering the development board results in the following console output:

(/home/korena/Development/Workspaces/car-dashboard/net_stack/host_scripts/) C-Kermit>connect
Connecting to /dev/ttyUSB0, speed 115200  
 Escape character: Ctrl-\ (ASCII 28, FS): enabled
Type the escape character followed by C to get back,  
or followed by ? to see other options.  
----------------------------------------------------
UART 0 Initialization complete ...

calling iROM copy function ...

Copying BL2 started ...

BL2 loading successful, running it ...

bl2 executing ...
                 Clock System initialization complete ...

memory initialization complete ...

Setting up timer ...

The INT_CSTAT register value is: 0x1

the value of TCFG register is:0x4000

the value of TCON register is:0x1

About to initialize ethernet networking ...

--- net_loop Entry

eth_start, calling net_init

initializing ARP ...

Done initializing ARP ...

Reseting ethernet device ...

resetting DM9000

resetting the DM9000, 1st reset

resetting the DM9000, 2nd reset

Done resetting device, probing ... 

dm9000 i/o: 0x88000000, id: 0x90000A46 

Device successfully probed ...

detecting bit mode ...

dumping registers .. 

NCR   (0x00): 0

NSR   (0x01): 0

TCR   (0x02): 0

TSRI  (0x03): 0

TSRII (0x04): 0

RCR   (0x05): 0

RSR   (0x06): 0

BPTR  (0x08): 37

ISR   (0xFE): 0

IMR   (0xFF): 0

FCR   (0x0A): 0

done dumping registers 

DM9000: running in 16 bit mode

MAC : [ :0::12::34::56::80::49:]

reading MAC from hardware ...

0

:12

:34

:56

:80

:49

:reading multicast from hardware ...

FF

:FF

:FF

:FF

:FF

:FF

:FF

:FF

:Activating DM9000

Enabling Interrupt ...

dm9000_phy_read(0x1): 0x7869

dm9000_phy_read(0x11): 0x8018

operating at 100M full duplex mode

eth_current successfully registered

dm9000_initialization complete.

--- NetState set to 0

--- net_loop Init

acting on chosen protocol ...

looping and polling ethernet recv ...

there's a lot of noise in this output due to debugging and stuff, but line 124 shows that auto negotiation of speed was achieved, so this is a pointer to the fact that we did indeed pass the second point in our test passing criteria above, to be sure, let's check the output from the host machine's side:

After connecting the cable, but before powering the development board:

korena@korena-solid:~/Development/Workspaces/car-dashboard/net_stack (master)$ sudo ethtool eth0
Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Half 1000baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Half 1000baseT/Full 
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: Unknown
        Supports Wake-on: g
        Wake-on: g
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: no

After powering the development board:

korena@korena-solid:~/Development/Workspaces/car-dashboard/net_stack (master)$ sudo ethtool eth0
Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Half 1000baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Half 1000baseT/Full 
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Link partner advertised link modes:  10baseT/Half 10baseT/Full 
                                             100baseT/Half 100baseT/Full 
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Speed: 100Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: off
        Supports Wake-on: g
        Wake-on: g
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes

So our board wasn't lying when it claimed successful auto-negotiation, and operating at 100Mb/s.

NOTE: For the purpose of the stuff we're doing in this post, I configured a static IP connection in the host machine, assigning the IP 10.0.0.20 to the host NIC, you probably know how to do this on a Linux machine.

receiving data

Now that we know our initialization routine works, let's work on receiving frames over the wire, we still won't be processing data, all we want to achieve is to have a working low level receiving function.
As soon as our host machine establishes connection, it starts shooting stuff left and right, mostly ARP requests and a flood of application specific stuff, we're not looking to decipher this data at our board's end, we're only trying to see if our low level receiving function can actually capture the data from the rx FIFO buffer in the DM9000 chip and pass it up through our eth.c (abstraction layer) to our net.c (protocol management layer) for processing. So let's look at our dm9000_rx function :

/*
 *   Received a packet and pass to upper layer
 *   */
static int dm9000_rx(struct eth_device *netdev)
{
        uint8_t rxbyte, *rdptr = (uint8_t *) net_rx_packets[0];
        uint16_t RxStatus=0, RxLen = 0;
        struct board_info *db = &dm9000_info;

        /* Check packet ready or not, we must check
         *         the ISR status first for DM9000A */

        if (!(DM9000_ior(DM9000_ISR) & 0x01)){ /* Rx-ISR bit must be set. */
                return 0;
        }

        DM9000_iow(DM9000_ISR, 0x01); /* clear PR status latched in bit 0 */

        /* There is _at least_ 1 package in the fifo, read them all */
        for (;;) {
                DM9000_ior(DM9000_MRCMDX);      /* Dummy read */
                /* Get most updated data,
                 *                 only look at bits 0:1, See application notes DM9000 */
                rxbyte = DM9000_inb(DM9000_DATA) & 0x03;
                /* Status check: this byte must be 0 or 1 */
                if (rxbyte > DM9000_PKT_RDY) {
                        print_format("DM9000 error: status check fail: 0x%x\n\r",
                                        rxbyte);
                        DM9000_iow(DM9000_RCR, 0x00);   /* Stop Device */
                        DM9000_iow(DM9000_ISR, 0x80);   /* Stop INT request */
                        // reset device ...
                        dm9000_reset();
                        dm9000_start();
                        return 0;
                }

                if (rxbyte != DM9000_PKT_RDY){
                        print_format("no packets received\n\r");
                        return 0; /* No packet received, ignore */
                }



                DM9000_DBG("receiving packet\n\r");
                //udelay(20);
                /* A packet ready now  & Get status/length */
                (db->rx_status)(&RxStatus, &RxLen);
                

                DM9000_DBG("rx status: 0x%x rx len: %d\n\r", RxStatus, RxLen);

                /* Move data from DM9000 */
                /* Read received packet from RX SRAM */
                (db->inblk)(rdptr, RxLen);

                DM9000_DBG("net_rx_packets filled ...\n\r");

                /*
                * a good RxStatus would look like: 0x4001
                * higher byte 0x40 is being anded with 0xBF, which should
                * produce 0, unless something is wrong ...
                */
                if ((RxStatus & 0xbf00) || (RxLen < 0x40)
                                || (RxLen > DM9000_PKT_MAX)) {
                        if (RxStatus & 0x100) {
                                print_format("rx fifo error\n\r");
                                /*debugging ... I want to actually see what's received !!!*/
                                dm9000_dump_regs();
                                dm9000_dump_eth_frame(rdptr,RxLen);
                        }
                        if (RxStatus & 0x200) {
                                print_format("rx crc error\n\r");
                        }
                        if (RxStatus & 0x8000) {
                                print_format("rx length error\n\r");
                        }
                        if (RxLen > DM9000_PKT_MAX) {
                                print_format("rx length too big\n\r");
                                dm9000_reset();
                                dm9000_start();
                                //      dm9000_dump_regs();
                        }
                } else {
                        DM9000_DBG("passing packet to upper layer\n\r");
                                dm9000_dump_eth_frame(rdptr,RxLen);
                        return RxLen;
                }
        }
        return 0;
}

The way things are done here are pretty easy to follow, especially if you keep a close look at the DM9000AEP application note, so we're not going to detail this process. We will focus on the function dm9000_dump_eth_frame(...) :

void dm9000_dump_eth_frame(uint8_t *packet,int len){
        print_format("ethernet header print ------\n\r");
        print_format("1 byte is: 0x%x\n\r",packet[0]);
        print_format("2 byte is: 0x%x\n\r",packet[1]);
        print_format("3 byte is: 0x%x\n\r",packet[2]);
        print_format("4 byte is: 0x%x\n\r",packet[3]);
        print_format("5 byte is: 0x%x\n\r",packet[4]);
        print_format("6 byte is: 0x%x\n\r",packet[5]);
        print_format("7 byte is: 0x%x\n\r",packet[6]);
        print_format("8 byte is: 0x%x\n\r",packet[7]);
        print_format("9 byte is: 0x%x\n\r",packet[8]);
        print_format("10 byte is: 0x%x\n\r",packet[9]);
        print_format("11 byte is: 0x%x\n\r",packet[10]);
        print_format("12 byte is: 0x%x\n\r",packet[11]);
        print_format("13 byte is: 0x%x\n\r",packet[12]);
        print_format("14 byte is: 0x%x\n\r",packet[13]);
        print_format("15 byte is: 0x%x\n\r",packet[14]);
        print_format("16 byte is: 0x%x\n\r",packet[15]);
        print_format("17 byte is: 0x%x\n\r",packet[16]);
        print_format("18 byte is: 0x%x\n\r",packet[17]);
        print_format("19 byte is: 0x%x\n\r",packet[18]);
        print_format("20 byte is: 0x%x\n\r",packet[19]);
        print_format("21 byte is: 0x%x\n\r",packet[20]);
        print_format("ethernet header print end ------\n\r");
}

The dumping function is deliberately written this way for readability, don't do this.
We want to compare this dump to the actual data the host machine sends out, so let's get to it. After compiling and flashing and all that, a section of the console shows:

receiving packet

rx status: 0x4001 rx len: 64

net_rx_packets filled ...

passing packet to upper layer

ethernet header print ------

1 byte is: 0xFF

2 byte is: 0xFF

3 byte is: 0xFF

4 byte is: 0xFF

5 byte is: 0xFF

6 byte is: 0xFF

7 byte is: 0xE0

8 byte is: 0xDB

9 byte is: 0x55

10 byte is: 0xE7

11 byte is: 0x6D

12 byte is: 0x9D

13 byte is: 0x8

14 byte is: 0x6

15 byte is: 0x0

16 byte is: 0x1

17 byte is: 0x8

18 byte is: 0x0

19 byte is: 0x6

20 byte is: 0x4

21 byte is: 0x0

ethernet header print end ------

packet received

destination and source IPs

Receive from protocol 0x806

Got ARP

Some where in the source code, I placed a piece of code that will stop looping when any ARP request is detected:

print_format("Got ARP\n\r");

Don't worry about the place where this line of code is, we'll get to that later. For now, let's run a tcp dump command on the host machine to see what's sent:

korena@korena-solid:~/Development/Workspaces/car-dashboard/net_stack (master)$ sudo tcpdump -XX -v -i eth0

And the output corresponding to the console output we got above is :

14:53:28.232542 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has sc-in-f105.1e100.net tell 10.0.0.20, length 28
        0x0000:  ffff ffff ffff e0db 55e7 6d9d 0806 0001  ........U.m.....
        0x0010:  0800 0604 0001 e0db 55e7 6d9d 0a00 0014  ........U.m.....
        0x0020:  0000 0000 0000 4a7d 4469                 ......J}Di

You can see that we matched the first 21 bytes of this request. We can now move on to dissect the request and work with it in the higher stacks. You can go back to a couple of posts back, specifically to Bootloader Implementation (Milestone 4 - part 3), and you can follow what this packet dump means.

Conclusion

So in this post, we went through initialization testing, and testing of our port for the receiving functionality, next, we'll look into sending packets, but for us to do that, we'll go through the process of parsing the ARP requests from the host machine, then responding to it properly.