Hi all, I’m having an issue with accessing my headless system. It seems SSH and getty serial terminal both take a very long time (5-10 minutes) to start after boot. I’ve attched the SSH logs from systemctl but I’m not sure how helpful these are. I see no output until 5-10 minutes in, then I see all the below and SSH is available.
Any thoughts or suggestions on how I can debug this?
user@ubiquityrobot:~$ journalctl -u ssh
– Logs begin at Thu 2016-02-11 16:28:01 UTC, end at Tue 2020-09-22 21:50:43 UTC
Feb 11 16:33:04 ubiquityrobot systemd[1]: Starting OpenBSD Secure Shell server…
Feb 11 16:33:04 ubiquityrobot sshd[1532]: Server listening on 0.0.0.0 port 22.
Feb 11 16:33:04 ubiquityrobot sshd[1532]: Server listening on :: port 22.
Feb 11 16:33:04 ubiquityrobot systemd[1]: Started OpenBSD Secure Shell server.
Sep 22 21:41:59 ubiquityrobot systemd[1]: Reloading OpenBSD Secure Shell server.
Sep 22 21:41:59 ubiquityrobot sshd[1532]: Received SIGHUP; restarting.
Sep 22 21:41:59 ubiquityrobot systemd[1]: Reloaded OpenBSD Secure Shell server.
Sep 22 21:41:59 ubiquityrobot sshd[1532]: Server listening on 0.0.0.0 port 22.
Sep 22 21:41:59 ubiquityrobot sshd[1532]: Server listening on :: port 22.
There are a few things that can lead to it taking a couple minutes stuck in linux before it can move on and run our startup files.
I am going to let the person most familiar with this issue give you a better answer and am just going to mention some known issues.
Due to recent shipment restrictions we are unable to ship the CR2032 battery that runs the real time clock for the system. This realtime clock is located on the large main board or MCB as we call it.
Please put in a battery if you have not yet done that and then let the system start. The system needs to see an outside network and contact an NTP server then the realtime clock will be set.
One cause for extremely long startup is lack of time as the system boots.
I recall some other slow startup causes but do not have details so I will tell the other dev folks to get back to this thread to offer more information.
Thanks for the reply! This led me down some other paths and I ended up finding a similar post on here that unearthed that the issue was with networking timing out. After changing the timeout from 5 minutes to 10 seconds my services are available after just a 10 second delay. Not sure what the root causes is but this will save me a lot of headache.
I think you are talking about the solution here: ROS image on Raspberry Pi 4?. I am not sure what the root cause is, but I suspect it is waiting for a internet connection, and not just a local connection. A 10 second timeout is much more reasonable.
We are planning on having a new image this month that has that fix integrated.