Trying to boot my Raspberry Pi 4 from netboot

festuc · December 20, 2024, 2:53pm

Network Setup

Mikrotik router at 10.10.10.1, serving DHCP for 10.10.10.0/24.
A server at 10.10.10.11 with TFTP and NFS.

Attempts so far

1st Attempt: Using external TFTP server at 10.10.10.11

Configured the DHCP server on Mikrotik with:
Option 67 (boot-file-name): ‘start4.elf’
Option 66 (next-server): ‘10.10.10.11’ (raw value: 0a0a0a0b).
Verified that the Raspberry Pi receives:
VI_ADDR: 10.10.10.17
SI_ADDR: 10.10.10.11.
However, the Raspberry Pi does not download any files from the TFTP server at 10.10.10.11.
Verified that TFTP server works by downloading files from another computer without issues.

2nd Attempt: Using Mikrotik’s built-in TFTP server

Removed next-server configuration from the DHCP server and enabled Mikrotik’s TFTP:

/ip tftp print
    Columns: IP-ADDRESSES, REQ-FILENAME, REAL-FILENAME, ALLOW, READ-ONLY, ALLOW-ROLLOVER, HITS
    # IP-ADDRESSES   REQ-FILENAME  REAL-FILENAME  ALLOW  READ-ONLY  ALLOW-ROLLOVER  HITS
    0 10.10.10.0/24  .*            /usb1-part1/   yes    yes        no                22

Tried downloading files from Mikrotik’s TFTP server:
Using a TFTP client on my laptop, I can connect to 10.10.10.1.
Small files like cmdline.txt (84 bytes) download successfully.
Larger files like start4.elf (2,916.4 KiB) and config.txt (1,529 bytes) fail.
Adjusted max block size on the Mikrotik TFTP server:
Tried 512 and 1454, but neither resolved the issue.

Current Issues

Raspberry Pi receives DHCP Offer correctly (with boot-file-name and next-server), but:
It does not initiate a download from the TFTP server (start4.elf or any other file).
Mikrotik’s built-in TFTP server:
Works for small files.
Fails to serve larger files like start4.elf.

What I Need Help With

Why is the Raspberry Pi not downloading files from the TFTP server, even though it receives the correct SI_ADDR ?
How i check if boot-file-name is well recived by Raspberry Pi?
Are there any limitations or known issues with Mikrotik’s TFTP server for serving larger files?
Is there any specific configuration required on the Raspberry Pi for it to work with Mikrotik’s TFTP server?

Additional Notes

The Raspberry Pi bootloader has been updated to the latest version (April 15, 2024).
Routeros is 7.16.2
TFTP from Mikrotik’s server works partially (only for small files).
All configurations and tests were validated using tcpdump on both the Raspberry Pi and the server.

Steveocee · December 21, 2024, 1:31pm

Would it. It be easier to run dhcp from your boot server? Less moving parts.

Are you sure the bootloader in the pi is updated? As in have you got another working setup you can test on?

festuc · December 21, 2024, 3:46pm

Would it. It be easier to run dhcp from your boot server? Less moving parts.

Are you sure the bootloader in the pi is updated? As in have you got another working setup you can test on?
[/quote
]All my Raspberry Pi devices at home have updated firmware and EEPROM and are configured to attempt netboot. I use a Mikrotik (RouterOS 7.16.2) as the DHCP server and a separate TFTP and NFS server at 10.10.10.11. However, the Raspberry Pi doesn’t download the required boot files, even though the configuration seems correct.
It’s easy forgot dhcp from mikrotik, but my plan is use same rpi compressed file to load in memory at rpi booting in multiple locations accross vpn.
I do not need (want) have a tftp in each location the big part of compressed root filesystem can be get from internet. I have Mikrotik boards in each location, the best way for me is get all potential from mikrotik.
I made more test, and see what mikrotik send when ask for a dhcp client. I run wireshark and get this it doesn-t send 66 or 67 even i check forced on options declaration
my /ip dhcp diferent pages:

/ip dhcp-server network print detail
Flags: D - dynamic 
 0   address=10.10.10.0/24 gateway=10.10.10.1 dns-server=10.10.10.1 wins-server="" ntp-server=10.10.10.1 caps-manager="" 
     domain="festuc.info" next-server=10.10.10.11 boot-file-name="bootcode.bin" dhcp-option="" 
     /ip dhcp-server/option/print  
Columns: NAME, CODE, VALUE, RAW-VALUE
#  NAME            CODE  VALUE           RAW-VALUE               
0  next-server       66  '10.10.10.11'   0a0a0a0b                
1  boot-file-name    67  'bootcode.bin'  626f6f74636f64652e62696e
 /ip dhcp-server/lease/print detail 
Flags: X - disabled, R - radius, D - dynamic, B - blocked
...
7   address=10.10.10.17 mac-address=DC:A6:32:45:37:14 use-src-mac=yes client-id="ff:f8:ce:1b:a1:0:2:0:0:ab:11:b1:b1:86:4:53:e9:79:7c" 
     address-lists="" server=dhcp1 dhcp-option=next-server,boot-file-name dhcp-option-set=rpi status=bound expires-after=22m15s 
     last-seen=7m45s age=23h37m7s active-address=10.10.10.17 active-mac-address=DC:A6:32:45:37:14 active-server=dhcp1 host-name="s3" 
     class-id="PXEClient:Arch:00000:UNDI:002001" 
...
/ip dhcp-server/network/print  detail 
Flags: D - dynamic 
 0   address=10.10.10.0/24 gateway=10.10.10.1 dns-server=10.10.10.1 wins-server=10.10.10.111 ntp-server=10.10.10.1 caps-manager=10.1.1.1 
     domain="festuc.info" next-server=10.10.10.11 boot-file-name="bootcode.bin" dhcp-option=next-server,boot-file-name dhcp-option-set=rpi 

/ip dhcp-server/option/sets/ print    
Columns: NAME, OPTIONS
#  NAME  OPTIONS       
0  rpi   next-server   
         boot-file-name

I captured traffic with Wireshark and noticed that the Mikrotik doesn’t send DHCP options 66 (next-server) or 67 (boot-file-name), even though they are configured in multiple places.
Example capture:

Frame 38: 340 bytes on wire (2720 bits), 340 bytes captured (2720 bits) on interface eth0, id 0
    Section number: 1
    Interface id: 0 (eth0)
        Interface name: eth0
    Encapsulation type: Ethernet (1)
    Arrival Time: Dec 21, 2024 16:06:05.486634557 CET
    UTC Arrival Time: Dec 21, 2024 15:06:05.486634557 UTC
    Epoch Arrival Time: 1734793565.486634557
    [Time shift for this packet: 0.000000000 seconds]
    [Time delta from previous captured frame: 0.004416057 seconds]
    [Time delta from previous displayed frame: 0.004416057 seconds]
    [Time since reference or first frame: 573.134268643 seconds]
    Frame Number: 38
    Frame Length: 340 bytes (2720 bits)
    Capture Length: 340 bytes (2720 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:ethertype:ip:udp:dhcp]
    [Coloring Rule Name: UDP]
    [Coloring Rule String: udp]
Ethernet II, Src: s3 (dc:a6:32:45:37:14), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Destination: Broadcast (ff:ff:ff:ff:ff:ff)
        .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
        .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
    Source: s3 (dc:a6:32:45:37:14)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IPv4 (0x0800)
    [Stream index: 0]
Internet Protocol Version 4, Src: 0.0.0.0 (0.0.0.0), Dst: 255.255.255.255 (255.255.255.255)
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
    Total Length: 326
    Identification: 0x0000 (0)
    000. .... = Flags: 0x0
    ...0 0000 0000 0000 = Fragment Offset: 0
    Time to Live: 64
    Protocol: UDP (17)
    Header Checksum: 0x78e8 [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 0.0.0.0 (0.0.0.0)
    Destination Address: 255.255.255.255 (255.255.255.255)
    [Stream index: 0]
User Datagram Protocol, Src Port: bootpc (68), Dst Port: bootps (67)
    Source Port: bootpc (68)
    Destination Port: bootps (67)
    Length: 306
    Checksum: 0xb17d [unverified]
    [Checksum Status: Unverified]
    [Stream index: 0]
    [Stream Packet Number: 37]
    [Timestamps]
    UDP payload (298 bytes)
Dynamic Host Configuration Protocol (Request)
    Message type: Boot Request (1)
    Hardware type: Ethernet (0x01)
    Hardware address length: 6
    Hops: 0
    Transaction ID: 0x5e381f0d
    Seconds elapsed: 8
    Bootp flags: 0x0000 (Unicast)
        0... .... .... .... = Broadcast flag: Unicast
        .000 0000 0000 0000 = Reserved flags: 0x0000
    Client IP address: 0.0.0.0 (0.0.0.0)
    Your (client) IP address: 0.0.0.0 (0.0.0.0)
    Next server IP address: 0.0.0.0 (0.0.0.0)
    Relay agent IP address: 0.0.0.0 (0.0.0.0)
    Client MAC address: s3 (dc:a6:32:45:37:14)
    Client hardware address padding: 00000000000000000000
    Server host name not given
    Boot file name not given
    Magic cookie: DHCP
    Option: (53) DHCP Message Type (Request)
        Length: 1
        DHCP: Request (3)
    Option: (61) Client identifier
        Length: 19
        IAID: f8ce1ba1
        DUID Type: assigned by vendor based on Enterprise number (2)
        Enterprise-number: Tom Gundersen (systemd) (43793)
        Identifier: b1b1860453e9797c
    Option: (55) Parameter Request List
        Length: 11
        Parameter Request List Item: (1) Subnet Mask
        Parameter Request List Item: (3) Router
        Parameter Request List Item: (6) Domain Name Server
        Parameter Request List Item: (12) Host Name
        Parameter Request List Item: (15) Domain Name
        Parameter Request List Item: (33) Static Route
        Parameter Request List Item: (42) Network Time Protocol Servers
        Parameter Request List Item: (114) DHCP Captive-Portal
        Parameter Request List Item: (120) SIP Servers
        Parameter Request List Item: (121) Classless Static Route
        Parameter Request List Item: (162) Unassigned
    Option: (57) Maximum DHCP Message Size
        Length: 2
        Maximum DHCP Message Size: 1472
    Option: (54) DHCP Server Identifier (10.10.10.1)
        Length: 4
        DHCP Server Identifier: 10.10.10.1 (10.10.10.1)
    Option: (50) Requested IP Address (10.10.10.17)
        Length: 4
        Requested IP Address: s3 (10.10.10.17)
    Option: (12) Host Name
        Length: 2
        Host Name: s3
    Option: (255) End
        Option End: 255

Has you can see i put every 66 i 67 option in every place i can, but only one time in each space. Same results

festuc · December 21, 2024, 4:40pm

dhcp offer
i don t see dchp offer in the last reply now i see it but still have not success

jaclaz · December 21, 2024, 4:42pm

maybe useful, maybe not:
http://forum.mikrotik.com/t/liebert-is-webcard-hid9-ups-monitoring-card-dhcp-compatibility-issue/172372/6

Steveocee · December 22, 2024, 10:55am

Are you able to put the Pi on it’s own interface into the MikroTik? You can torch the interface and see what’s going on then during the boot process.
You should see dst 69’s for the TFTP going to it then maybe some dst 111’s if you’re booting from an image on nfs?

that should help to figure out where the problem is coming from.

xrtc · February 14, 2025, 3:41pm

Did you find solution to this ? I am trying on rpi5 with no luck