Cisco Debug Persists Through Reboot

Normal boot time messages from a C881 router look something like this:

 System Bootstrap, Version 15.4(1r)T, RELEASE SOFTWARE (fc1)  
 Technical Support: http://www.cisco.com/techsupport  
 Copyright (c) 2013 by cisco Systems, Inc.  
   
 Total memory size = 1024 MB  
 C881-K9      platform with 1048576 Kbytes of main memory  
 Main memory is configured to 32 bit mode   
   
 Readonly ROMMON initialized  
   
   
 IOS Image Load Test   
 ___________________   
 Digitally Signed Production Software   
 Self decompressing the image : ###<snip>### [OK]  
   

But there's one router in the fleet which does this instead:

 System Bootstrap, Version 15.4(1r)T, RELEASE SOFTWARE (fc1)  
 Technical Support: http://www.cisco.com/techsupport  
 Copyright (c) 2013 by cisco Systems, Inc.  
   
 Total memory size = 1024 MB  
 C881-K9      platform with 1048576 Kbytes of main memory  
 Main memory is configured to 32 bit mode  
   
 Readonly ROMMON initialized  
 Using monlib version 2  
 Using version info 2  
   
  dfs_openfile: Opening file.....  
  dfs_openfile: Opened file / with fib = 4019e5c  
 Reading cluster = 126, offset = 0, nsecs = 8  
 Reading cluster = 133, offset = 0, nsecs = 8  
 Reading cluster = 17013, offset = 0, nsecs = 8  
 Reading cluster = 17458, offset = 0, nsecs = 8  
 Reading cluster = 18056, offset = 0, nsecs = 8  
 Reading cluster = 17053, offset = 0, nsecs = 8  
  dfs_closefile: Closing file.... 4019e5c  
  dfs_closehandle: Closed file.... 4019e5cUsing monlib version 2  
 Using version info 2  
   
  dfs_openfile: Opening file....c800-universalk9-mz.SPA.154-3.M4.bin  
  dfs_closefile: Closing file.... 4019e5c  
  dfs_closehandle: Closed file.... 4019e5c  
  dfs_openfile: Opened file /c800-universalk9-mz.SPA.154-3.M4.bin with fib = 4019e5c  
 Reading cluster = 17053, offset = 0, nsecs = 8  
   
 Reading cluster = 17053, offset = 0, nsecs = 128  
 Reading cluster = 17069, offset = 0, nsecs = 128  
 Reading cluster = 17085, offset = 0, nsecs = 128  
 <--- ~1300 lines removed --->  
 Reading cluster = 58703, offset = 0, nsecs = 128  
 Reading cluster = 58719, offset = 0, nsecs = 80  
 IOS Image Load Test  
 ___________________  
 Digitally Signed Production Software  
 Self decompressing the image : ###<snip>### [OK]  

Um. Okay. What's going on here?

It turns out that there are a couple of funny things about debugging filesystem operations:

1. You can't see that it's turned on from within IOS:

C881#debug filesystem flash:
C881#show debugging




C881#



2. The setting persists across both reboots and power interruptions. Apparently this directive sets a hardware flag somewhere. no debug filesystem flash: made those messages go away.

The setting probably applies to any storage device available from within the ROMMON:

 rommon 1 > dev  
 Devices in device table:  
     id name  
   flash: compact flash         
 bootflash: boot flash           
 usbflash0: usbflash0           

This was new to me.

Let's Encrypt plugin for ASA

I wrote a certbot (Let's Encrypt ACME client) plugin for Cisco ASA. It runs on a separate box, talks ACME to the Let's Encrypt service and uses the ASA REST API to manage certificates on the ASA.

Details here.

If you try it out, please let me know how it goes?