Apparently early on Saturday morning there was a power outage for a flicker or something. I didn’t notice this at all, but Karishma did. Flash forward to today around 11:30, when we get this email from Celia, “I just brought my use down to 65092 MB, but I cannot access the linux machines. Any chance I could be given access again?” Scott responds asking if she is using the right password and still having trouble. She responds that she is using the right password and still having trouble. I arrive at the Science Center at ~2:30 pm.
Celia is in the Microfocus and I ask her if she’s still having trouble and what she’s tried. She is still having trouble and points to about 5 different client machines that she’s tried logging into. I ask if she can ssh into tempest. She successfully ssh’s into tempest. I ask her if she can ssh into puma. She cannot. Then I attempt logging into jay. It fails with edavis5, but I want to make sure Celia is able to do her work. I login to wren as luser, try to ssh into tempest, it fails (doesn’t know what tempest is). I become root@wren and ifup eth0. Then Celia is able to ssh into tempest and do her work.
In this debugging process, I run ah-broadcast ping (which I now realize is a bad thing to do with an ah-broadcast, ah-broadcast hostname would be much better, but this gave me the message I needed). None of the clients were up. This was a larger problem than just Celia and I not being able to login.
I logged in as luser to jay, brought eth0 back up, mounted all and rebooted. I was now able to login as edavis5. For finch, I did a hard reboot (pressing the button and then pressing it again). This didn’t work. I was then not able to login as edavis5, so I figured that the ifup eth0 and mount -a was necessary.
Unfortunately, I also have a life. So I had a meeting and church and was able to get back to the clients at 5 pm. Lulu joined me around 5:15. By the time Lulu had joined, I had brought eth0 up and rebooted on 5 machines, so there were only 3 left in the microfocus for her. But! When I then returned to Finch to ifup eth0 on Finch…I got “Trying to connect…is the cord connected?” which is a very nice error message if you ask me. I was expecting it to just fail and I would flail around to see what was wrong. Turns out that ethernet port was broken. Because I tried connecting my laptop to the internet using the other side connected to the port, I tried using a different cord but neither worked so it must be the port. I then proceeded to try all ports that weren’t already connected to a client that was working. There were not very many of these, but none of them worked either. This seemed crazy. All of the ethernet ports should work. Just like all of the power outlets should work…but they don’t.
Then Lulu told me that she had rebooted cardinal and swallow and irwin but none of them worked. I already knew that irwin wasn’t working because I had given a sticky note to it and had tried logging in on Wednesday but that didn’t work. But swallow I was also on on Wednesday and it should’ve worked with the reboot.
So I tried logging into them, and it worked for me. So we decided that there was another LDAP style problem with the hye account not working but my edavis5 account working. Why would this happen?? 🙁
Then Lulu rebooted all of the clients not in the microfocus. Thank you! While I tried to figure out how to tell the difference between ones that knew hye and ones that didn’t. I couldn’t figure anything else out, got frustrated and needed to eat dinner so I left.
So now the outstanding issues:
-why can edavis5 login when hye can’t?
-why is there an ethernet drop down?
-Then something that I noticed when trying to look into Celia’s account specifically, was that her entry in tempest’s /etc/passwd was different from her entry in puma’s /etc/passwd. Specifically in the type of shell– scponly v bash. Why would this be like that? Should we make sure those are the same?
-Also running ah-broadcast after Lulu brought some of the machines in 173 back up, we still need to copy the ssh keys for orangutan and tamarin over to tempest, because when I was ah-broadcasting it asked for root@orangutan and root@tamarin’s passwords. This shouldn’t be hard and I have a blog post about it from before.