Varnish

Two very very late posts about what to do if you think you screwed up a config file or other system file (you do not necessarily an update — might affect other dependencies) and how to set up varnish.

Part I: What to do if you think you screwed up a config file

When we attempted to download varnish on 11/8, we hit some issues with our httpd.conf file that was just outdated/incorrect (for reasons unknown, problem revealed later). For context, on our machines, we edit the httpd.conf file in the sysadmin account and use a script to update the tempest server’s httpd.conf.

Our problem: When we tried to restart httpd, we got an error that it “cannot load httpd/modules/mod_file_cache.so.” We had restarted apache the previous Wednesday to test out varnish on puma, so this was peculiar.

Scott did some detective work in the rsnapshots since April, saw that the other httpd.conf files hadn’t needed to load mod_file_cache.so. After grepping through the yum.logs, we saw that httpd had last updated on october 14th. Did this actually help? We were still confused.

We first tried just commenting out the module in httpd.conf. Then it failed and wanted another module. Trying again: we thought maybe we should just redownload a copy of httpd.conf (using yum reinstall). However, we were concerned that yum reinstall would overwrite the entire httpd directory: there were some (not many) Wellesley specific edits (wellesley.conf) that we didn’t want to overwrite, so we backed them up in the sysadmin account…but then found this article http://www.cyberciti.biz/faq/yum-downloadonly-plugin/! The yum downloadonly flag makes yum only download the package and not install/update them.

>>yumdownloader -v httpd

which puts the httpd files in the current directory. After moving them to /var/tmp/, we used rpm2cpio to extract the files. Ideally, we thought this way we would be able to minimize the amount of edits transferring needed. To untar, we used cpio -idmv:

d: make directories

i: extract

m: preserve modification times

v: vebose

After trying this out, it created subdirectory /usr/var/etc. We looked into etc and saw httpd and logrotate.d and sysconfig. We went into httpd/conf, and saw a httpd.conf file, last updated August 15th.

Interesting: we diff’d the new file and our current httpd.conf (the weird reverted version) and saw that there was a lot of student stuff that had been removed before. Purely looking at the size of the different httpd.conf files, the one we just downloaded using yum –downloadonly was ~35k (as were the rsnapshot daily backups we looked at before) while the current httpd.conf file was ~55k, this confirmed that the version of httpd.conf was odd. Hmm. We ended up just copying the rsnapshot version of httpd.conf (since the snapshot versions had the Wellesley conf edits) into the sysadmin/etc/ version. Comparing the rsnapshot version with the newly downloaded version, confirmed that they weren’t all that different, so we tested, it worked, and we proceeded to use Scott’s httpd.conf update script to update the httpd.conf

>> service httpd start

*now works!

After this side journey in fixing the httpd.conf file, we then cleaned up the /var/tmp (can delete the usr and var and etc), and we kept the yum –downloadonly file so that we can always cpio it again if we want some more info.

So takeaways from the mystery httpd.conf file problem: use yum –downloadonly if necessary so you don’t overwrite your entire directory, use cpio to untar (know that yum –download only downloads into the current directory that you are in). Have an excellent week!

Part II: Setting up varnish

In light of some recent research completions, there were some concerns about whether tempest could handle the brunt of requests hitting the server. After some weeks of exploration, we decided to try out varnish and set it up on tempest. We’re still keeping track of whether varnish will work best for our system, but this is what we did to configure it:

  1. Edit /etc/varnish/default.vcl

We changed the backend default to be port 800 since we wanted varnish pointing to 127.0.0.1:800.

  1. We changed apache to be listening on port 127.0.0.1:800 on /etc/httpd/conf/httpd.conf to listen to varnish.

So what we have now is we’ve set up varnish backend to point to port 800 and for apache to see varnish on port 800.

  1. Now we need to set varnish to listen on port 80 (where apache used to be). To do this, we configured /etc/sysconfig/varnish to alternative option 3. We hit a problem when trying to test this out on puma where we forgot to comment out another alternative option. Doing that will make varnish not work.

    1. minor edit: we set the varnish cache file size from its default 256M to 1GB

After doing >>service varnish start, it seemed to work.

Note: you can’t test by having varnish and apache going at the same time(if both listen to port 80…), so have to restart apache first (Won’t be listening on port 80) then start varnish to listen on port 80.

  1. We tested varnish to see if it impacted our requests by using >>GET -Used http://cs.wellesley.edu/~cs110 to see what happens when we request a file. To check varnish stats, look at… >>varnishstats which will show the cache hits. When testing it on puma, we also used >>nmap puma to see what was on port 80.

So excellent! Varnish works. We’ll probably have another post soon about observed changes with varnish.

Posted in Uncategorized | Leave a comment

robots.txt

A few months ago, we talked about adding a robots.txt file so that webcrawlers won’t index archived courses, such as the course material for cs110 back in fall of 2012.

The robotstxt.org page is, unsurprisingly, the authority on creating a robots.txt file. You put the robots.txt file in the top level directory of the webserver, which on tempest is /var/www/html. When I looked, there was actually already a robots.txt page, so I just added a few lines like  ”Disallow:/~cs110f12/”. Hopefully this will prevent robots from indexing those archived courses.

Posted in Uncategorized | Leave a comment

Installing HTTPS certificates

To create a signed certificate, we first ran genkey and created a file that I happened to call cs.wellesley.edu.1.csr.  I put that in /etc/pki/tls/certs.

When I uploaded it to the certificate company’s website, they complained that it didn’t meet the 2048 bit minimum, so I created a second as cs.wellesley.edu.2.csr

After they did their work, I got an email back that looked like this:

* Click the following link to download your SSL certificate (generally try to use a version that includes intermediates & root or your certificate may be rejected by some older clients)

Format(s) most suitable for your server software:
as X509 Certificate only, Base64 encoded: https://cert-manager.com/customer/InCommon/ssl?action=download&sslId=12345678&format=x509CO
as X509 Intermediates/root only, Base64 encoded: https://cert-manager.com/customer/InCommon/ssl?action=download&sslId=12345678&format=x509IO

And no further information.  Thanks to help from Andy Maroney, I now know that these files are the targets of two config variables in /etc/httpd/conf.d/ssl.conf.  There’s also a third variable that needs to be configured.

  • The first one downloads as cs_wellesley_edu_cert.cer. I put it in /etc/pki/tls/certs and made it the value of SSLCertificateFile
  • The second one downloads as cs_wellesley_edu_interm.cer. I put it in /etc/pki/tls/certs and made it the value of SSLCertificateChainFile
  • The third variable is SSLCertificateKeyFile and the value of that is set to a file that was generated simultaneously with the original .csr file, and that is /etc/pki/tls/private/cs.wellesley.edu.key. (That file holds the private key and is readable only by root.)

Restart Apache and you’re done!

Posted in Uncategorized | Leave a comment

Decreasing Logwatch Output

In the recent past, logwatch has been spitting out way, way too much info about activity on tempest. The output that the cs-sysadmin email receives is actually truncated. The main culprit is HTTP, which spits out every link that gets a 401, 403, and 404 error. On puma, we’ve turned off http messages completely, but this time we wanted to figure out a more refined way to filter messages.

I found the documentation for logwatch in /usr/share/doc. The default configuration files are found in the directory /usr/share/logwatch/default.conf/services for services such as http, sshd, pam_unix, etc. The default scripts are in /usr/share/logwatch/scripts/services. These scripts are all written in Perl.

To override the default configuration, the documentation recommends 2 ways. You can add in whatever overriding variables in the /etc/logwatch/conf directory in a file called logwatch.conf. Or you could copy the default from /usr/share/logwatch to the corresponding directory in /etc/logwatch, and modify the file in the /etc/logwatch folder.

You could set variables in the /etc/logwatch/conf directory. On puma, this is how we turned off the output for http in the logwatch.conf file, since the output was getting voluminous. However, the configuration files did not give us fine enough control over the output. I had to modify the scripts themselves.

To modify scripts, I copied the appropriate default script from /usr/share/logwatch/scripts/services over to /etc/logwatch/scripts/services, where I fiddled around with the output. For example for the http script, I changed it so that it prints out the total number of 404 errors tempest collects for the day, instead of every single URL.

In general, to configure logwatch to your liking, copy the original file from /usr/share/logwatch to its corresponding place in /etc/logwatch, and make your edits to the /etc/logwatch file. The output will be much more readable.

Sources: http://serverfault.com/questions/293226/linux-logwatch8-is-too-noisy-how-can-i-control-the-noise-level/293233#293233

https://www.digitalocean.com/community/tutorials/how-to-install-and-use-logwatch-log-analyzer-and-reporter-on-a-vps

 

 

Posted in Uncategorized | Leave a comment

CentoOS Reinstall Adventures – Password Trouble

Karina and I sat down in 173 this evening to make sure that the clients are working. We’re pretty sure orangutan, gorilla, gibbon, and tamarin are all good to go, and we’re currently installing centos on chimp.
However, when I log into these machines, I can’t log in with my new password. My old password gets me in fine, and my new one doesn’t! I thought we resolved this issue when I ran the password-change script on tempest a couple weeks ago…
In our start-up notes, we did say “no” to using auth config, then “yes” to flat-files, keeping in mind that we need to do authentication later. Perhaps this is related to my login issues?

 

Posted in Uncategorized | Leave a comment

cp -al taking a long time

Our backups on Puma have been taking a long time, finishing late in the day, almost in time for the next backup.  The problem seemed to be in the cp -al step, not in the rsync step.  I investigated, looking at how long the cp -al on each directory took, using code like this:

for a in `ls -A $from`; do
now=`date +%T`
echo "$now cp -al $from/$a to $to/$a"
cp -al $from/$a $to/$a
done

and the result looked like:

16:38:06 cp -al 2014-03-26/alice to today/alice
16:38:06 cp -al 2014-03-26/anderson to today/anderson
16:39:37 cp -al 2014-03-26/apache-tomcat-5.5.26 to today/apache-tomcat-5.5.26
16:39:42 cp -al 2014-03-26/appinvstats to today/appinvstats
22:59:14 cp -al 2014-03-26/appinv-stats to today/appinv-stats
22:59:14 cp -al 2014-03-26/btjaden to today/btjaden
23:11:11 cp -al 2014-03-26/compbio to today/compbio
23:11:12 cp -al 2014-03-26/cs to today/cs
23:11:24 cp -al 2014-03-26/cs110f11 to today/cs110f11
...

It finished at about midnight (so, less than 8 hours total), but essentially all of that time was in the appinvstats directory.

Sure enough, some subdirectories of that account had a *lot* of inodes.  Here are some useful references:

http://unixetc.co.uk/2012/05/20/large-directory-causes-ls-to-hang/

http://www.olark.com/spw/2011/08/you-can-list-a-directory-with-8-million-files-but-not-with-ls/

http://www.pronego.com/helpdesk/knowledgebase.php?article=59

A count of the inodes:

ls -fR collectedStats | wc -l
5057712
ls -fR errorFiles | wc -l
4438339

So, about 1 million files/folders or inodes.

Turning these in to tarfiles would reduce these to two inodes.  There’s also a savings in space:

du -csh errorFiles errorFiles.tar
129G    errorFiles
106G    errorFiles.tar
du -csh collectedStats collectedStats.tar
11G     collectedStats
3.3G       collectedStats.tar

We’ll look at replacing these directories with the tar files.

 

Posted in Uncategorized | Leave a comment

Install Python2.7 w/ Tkinter

Rhys asked for us to install Tkinter for Python2.7.  [Remember, that Python2.6 is the default Python on Tempest because it's what RHEL comes with, but most users are using Python 2.7.]  Here’s the current behavior:

[anderson@tempest ~] python2.6
Python 2.6.6 (r266:84292, Nov 21 2013, 10:50:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from Tkinter import *
>>> 
[anderson@tempest ~] python2.7
Python 2.7 (r27:82500, Sep 20 2012, 17:09:01) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from Tkinter import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/lib-tk/Tkinter.py", line 39, in <module>
    import _tkinter # If this fails your Python may not be configured for Tk
ImportError: No module named _tkinter
>>>

Which is obviously no good.  This seemed pretty straightforward, since the fact that it works for Python2.6 suggests that I have all the necessary .so files and such.  However, I couldn’t find any pre-compiled binaries for Python2.7 for RHEL6/CentOS6.  I tried compiling Python for myself, just like I did last August (https://blogs.wellesley.edu/cssysadmin/2012/08/29/python-2-7/  However, that resulted in exactly the same ImportError.

Here’s the best of the “how to compile Python” info pages that I found.  However, I tried one more time, but this time, I discovered a comment that explained that I have to install tk-devel.  That, however, is a piece of cake.

# yum -y install tk-devel

To avoid any issue with over-writing the existing Python2.7, which is installed in /usr/local/, I installed the new, improved, Python2.7 in /opt:

# wget http://python.org/ftp/python/2.7.6/Python-2.7.6.tar.xz
# tar xf Python-2.7.6.tar.xz
# cd Python-2.7.6
# ./configure --prefix=/opt --enable-unicode=ucs4 --enable-shared LDFLAGS="-Wl,-rpath /opt/lib"
# make && make altinstall

The first time I tried that, it failed miserably:

checking build system type... x86_64-unknown-linux-gnu
 checking host system type... x86_64-unknown-linux-gnu
 checking for --enable-universalsdk... no
 checking for --with-universal-archs... 32-bit
 checking MACHDEP... linux2
 checking EXTRAPLATDIR...
 checking for --without-gcc... no
 checking for gcc... gcc
 checking whether the C compiler works... no
 configure: error: in `/var/tmp/install-python2.7/Python-2.7.6':
 configure: error: C compiler cannot create executables
 See `config.log' for more details

It turned out that the trouble was that there wasn’t a /opt/lib directory.  Doing a “mkdir /opt/lib” and re-doing the config command worked perfectly, as did the “make” and “make altinstall”

So, now we have:

Python 2.7.6 (default, Mar 24 2014, 13:29:51)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from Tkinter import *
>>>

But, we also need to make sure that the new Python2.7 has all the libraries that the old Python2.7 had.

I learned that Pip can tell you the libraries it has loaded:

# /usr/local/bin/pip-2.7 freeze
CouchDB==0.9
MySQL-python==1.2.4b5
Orange==2.5a4
anyjson==0.3.3
beautifulsoup4==4.3.2
biopython==1.63
couchdbkit==0.6.5
dumptruck==0.1.5
gensim==0.8.6
numpy=1.6.2
scikit-learn=0.12.1
...

Compare to:

# /opt/bin/pip2.7 freeze
wsgiref==0.1.2

We can dump the former into a file and use it to set up the new Python:

# umask 0002
# /opt/bin/pip2.7 freeze
wsgiref==0.1.2
# rm old-py2.7-modules
# /usr/local/bin/pip-2.7 freeze > old-py2.7-modules
# /opt/bin/pip2.7 install -r old-py2.7-modules
Downloading/unpacking CouchDB==0.9 (from -r old-py2.7-modules (line 1))
Downloading CouchDB-0.9.tar.gz (55kB): 55kB downloaded
Running setup.py (path:/tmp/pip_build_root/CouchDB/setup.py) egg_info for package CouchDB

Downloading/unpacking MySQL-python==1.2.4b5 (from -r old-py2.7-modules (line 2))
Downloading MySQL-python-1.2.4b5.tar.gz (82kB): 82kB downloaded
...

Well, that *mostly* worked.  We ran into trouble after a while:

Downloading/unpacking scikit-learn==0.12.1 (from -r old-py2.7-modules (line 27))
Downloading scikit-learn-0.12.1.tar.gz (3.0MB): 3.0MB downloaded
Running setup.py (path:/tmp/pip_build_root/scikit-learn/setup.py) egg_info for package scikit-learn
Partial import of sklearn during the build process.
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/tmp/pip_build_root/scikit-learn/setup.py", line 36, in <module>
from numpy.distutils.core import setup
ImportError: No module named numpy.distutils.core
Complete output from command python setup.py egg_info:
Partial import of sklearn during the build process.

Traceback (most recent call last):

File "<string>", line 17, in <module>

File "/tmp/pip_build_root/scikit-learn/setup.py", line 36, in <module>

from numpy.distutils.core import setup

ImportError: No module named numpy.distutils.core

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/scikit-learn
Traceback (most recent call last):
File "/opt/bin/pip2.7", line 9, in <module>
load_entry_point('pip==1.5.4', 'console_scripts', 'pip2.7')()
File "/opt/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg/pip/__init__.py", line 185, in main
return command.main(cmd_args)
File "/opt/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg/pip/basecommand.py", line 161, in main
text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 70: ordinal not in range(128)

How much got installed?  Let’s see:

# /opt/bin/pip2.7 freeze
wsgiref==0.1.2

Well, that stinks! Is it all or nothing?  The error output suggests that there may be a dependency on numpy, so let’s install that first:

# /opt/bin/pip2.7 install numpy
# /opt/bin/pip2.7 freeze
numpy==1.8.0
wsgiref==0.1.2

Okay, and I notice that the version is newer than the numpy in the old Python2.7. So, this will be annoying and tedious, but I guess I may have to install things by hand. One more try:

# /usr/local/bin/pip-2.7 freeze | cut -d= -f1 | grep -v numpy > old-py2.7-module-names
# /opt/bin/pip2.7 install -r old-py2.7-module-names

Wait a *long* time, and it succeeds!

# diff old-py2.7-module-names new-py2.7-module-names
15a16
> numpy
26a28
> scipy

Woo hoo!

Note that pip can be used to upgrade Python packages.

Posted in Uncategorized | Leave a comment

March 15? Power outage…and troubles.

Apparently early on Saturday morning there was a power outage for a flicker or something. I didn’t notice this at all, but Karishma did. Flash forward to today around 11:30, when we get this email from Celia, “I just brought my use down to 65092 MB, but I cannot access the linux machines. Any chance I could be given access again?” Scott responds asking if she is using the right password and still having trouble. She responds that she is using the right password and still having trouble. I arrive at the Science Center at ~2:30 pm.

Celia is in the Microfocus and I ask her if she’s still having trouble and what she’s tried. She is still having trouble and points to about 5 different client machines that she’s tried logging into. I ask if she can ssh into tempest. She successfully ssh’s into tempest. I ask her if she can ssh into puma. She cannot. Then I attempt logging into jay. It fails with edavis5, but I want to make sure Celia is able to do her work. I login to wren as luser, try to ssh into tempest, it fails (doesn’t know what tempest is). I become root@wren and ifup eth0. Then Celia is able to ssh into tempest and do her work.

In this debugging process, I run ah-broadcast ping (which I now realize is a bad thing to do with an ah-broadcast, ah-broadcast hostname would be much better, but this gave me the message I needed). None of the clients were up. This was a larger problem than just Celia and I not being able to login.

I logged in as luser to jay, brought eth0 back up, mounted all and rebooted. I was now able to login as edavis5. For finch, I did a hard reboot (pressing the button and then pressing it again). This didn’t work. I was then not able to login as edavis5, so I figured that the ifup eth0 and mount -a was necessary.

Unfortunately, I also have a life. So I had a meeting and church and was able to get back to the clients at 5 pm. Lulu joined me around 5:15. By the time Lulu had joined, I had brought eth0 up and rebooted on 5 machines, so there were only 3 left in the microfocus for her. But! When I then returned to Finch to ifup eth0 on Finch…I got “Trying to connect…is the cord connected?” which is a very nice error message if you ask me. I was expecting it to just fail and I would flail around to see what was wrong. Turns out that ethernet port was broken. Because I tried connecting my laptop to the internet using the other side connected to the port, I tried using a different cord but neither worked so it must be the port. I then proceeded to try all ports that weren’t already connected to a client that was working. There were not very many of these, but none of them worked either. This seemed crazy. All of the ethernet ports should work. Just like all of the power outlets should work…but they don’t.

Then Lulu told me that she had rebooted cardinal and swallow and irwin but none of them worked. I already knew that irwin wasn’t working because I had given a sticky note to it and had tried logging in on Wednesday but that didn’t work. But swallow I was also on on Wednesday and it should’ve worked with the reboot.

So I tried logging into them, and it worked for me. So we decided that there was another LDAP style problem with the hye account not working but my edavis5 account working. Why would this happen?? :(

Then Lulu rebooted all of the clients not in the microfocus. Thank you! While I tried to figure out how to tell the difference between ones that knew hye and ones that didn’t. I couldn’t figure anything else out, got frustrated and needed to eat dinner so I left.

So now the outstanding issues:

-why can edavis5 login when hye can’t?

-why is there an ethernet drop down?

-Then something that I noticed when trying to look into Celia’s account specifically, was that her entry in tempest’s /etc/passwd was different from her entry in puma’s /etc/passwd. Specifically in the type of shell– scponly v bash. Why would this be like that? Should we make sure those are the same?

-Also running ah-broadcast after Lulu brought some of the machines in 173 back up, we still need to copy the ssh keys for orangutan and tamarin over to tempest, because when I was ah-broadcasting it asked for root@orangutan and root@tamarin’s passwords. This shouldn’t be hard and I have a blog post about it from before.

Posted in Uncategorized | Leave a comment

Install Tomcat 6 on RHEL 6

I’m following the instructions here: http://newpush.com/how-to-install-tomcat-6-on-rhel-6-or-centos-6/

First up, install yum-priorities. That allows us to prioritize some repos over others. So, for example, we can make sure the RHEL repos take priority over EPEL, I guess.  He also installs two repo files (enabling those repos) and a jpackage-utils (but not the jpackage repo).  Hmm.

I’ve typically gotten the effect of yum-priorities by leaving EPEL disabled, and enabling it when desired by doing:

yum –enablerepo=epel search foo

I’m going to skip that, then, and just search for the three packages he specifies:

[root@tempest ~] yum –enablerepo=epel,rpmforge search tomcat6 tomcat6-webapps tomcat6-admin-webapps
rhel-6-server-cf-tools-1-rpms                                                                | 2.8 kB     00:00
rhel-6-server-rhev-agent-rpms                                                                | 3.1 kB     00:00
rhel-6-server-rpms                                                                           | 3.7 kB     00:00
rpmforge                                                                                     | 1.9 kB     00:00
rpmforge/primary_db                                                                          | 2.7 MB     00:05
=============================================== N/S Matched: tomcat6 ===============================================
glite-security-trustmanager-tomcat6.noarch : Java trustmanager interface supporting a GSI grid name space
tomcat6.noarch : Apache Servlet/JSP Engine, RI for Servlet 2.5/JSP 2.1 API
tomcat6-el-2.1-api.noarch : Expression Language v1.0 API
tomcat6-jsp-2.1-api.noarch : Apache Tomcat JSP API implementation classes
tomcat6-lib.noarch : Libraries needed to run the Tomcat Web container
tomcat6-servlet-2.5-api.noarch : Apache Tomcat Servlet API implementation classes

Name and summary matches mostly, use “search all” for everything.
Warning: No matches found for: tomcat6-webapps
Warning: No matches found for: tomcat6-admin-webapps
[root@tempest ~]

Hmm.  Now I’m puzzled. The two missing packages aren’t in EPEL or in RPMFORGE, yet those are the only repositories he enables.  Very strange.  Let’s take a leap and just slavishly do as he tells us:

[root@tempest ~] yum install yum-priorities
No package yum-priorities available.
Error: Nothing to do
[root@tempest ~]

I did some searching and it turns out that on CentOS 6, It bothers me that the instructions fail at the first step.  On the other hand, this page: http://wiki.centos.org/PackageManagement/Yum/Priorities says that it should exist, which I find very bothersome:

[root@tempest ~] yum –enablerepo=epel search yum
Loaded plugins: downloadonly, product-id, refresh-packagekit, security, subscription-manager
This system is receiving updates from Red Hat Subscription Management.
rhel-6-server-cf-tools-1-rpms                                                                | 2.8 kB     00:00
rhel-6-server-rhev-agent-rpms                                                                | 3.1 kB     00:00
rhel-6-server-rpms                                                                           | 3.7 kB     00:00
================================================= N/S Matched: yum =================================================
anaconda-yum-plugins.noarch : Installation-related yum plugins
yum-arch.noarch : Extract headers from rpm in a old yum repository
yum-metadata-parser.x86_64 : A fast metadata parser for yum
yum-presto.noarch : Presto plugin for yum
yum-rhn-plugin.noarch : RHN support for yum
yumex.noarch : Yum Extender graphical package management tool
PackageKit-yum.x86_64 : PackageKit YUM backend
PackageKit-yum-plugin.x86_64 : Tell PackageKit to check for updates when yum exits
fusioninventory-agent-yum-plugin.noarch : Ask FusionInventory agent to send an inventory when yum exits
kabi-yum-plugins.noarch : The Red Hat Enterprise Linux kernel ABI yum plugin
repoview.noarch : Creates a set of static HTML pages in a yum repository
yum-dellsysid.x86_64 : YUM plugin to retrieve the Dell System ID
yum-plugin-aliases.noarch : Yum plugin to enable aliases filters
yum-plugin-changelog.noarch : Yum plugin for viewing package changelogs before/after updating
yum-plugin-downloadonly.noarch : Yum plugin to add downloadonly command option
yum-plugin-protect-packages.noarch : Yum plugin to prevents Yum from removing itself and other protected packages
yum-plugin-security.noarch : Yum plugin to enable security filters
yum-plugin-tmprepo.noarch : Yum plugin to add temporary repositories
yum-plugin-verify.noarch : Yum plugin to add verify command, and options
yum-plugin-versionlock.noarch : Yum plugin to lock specified packages from being updated
yum-utils.noarch : Utilities based around the yum package manager
createrepo_c.x86_64 : Creates a common metadata repository
grinder.noarch : A tool for synchronizing content from yum repositories
mash.noarch : Koji buildsystem to yum repository converter
mrepo.noarch : A tool to set up a yum/apt mirror from various sources
remi-release.noarch : YUM configuration for remi repository
yum.noarch : RPM installer/updater

Name and summary matches only, use “search all” for everything

 

I’m going to skip yum-priorities and go on to the rest of step one:

[root@tempest ~] rpm -Uvh http://apt.sw.be/redhat/el6/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
Retrieving http://apt.sw.be/redhat/el6/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
Preparing…                ########################################### [100%]
package rpmforge-release-0.5.2-2.el6.rf.x86_64 is already installed
[root@tempest ~] rpm -Uvh http://download.fedora.redhat.com/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm
Retrieving http://download.fedora.redhat.com/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm
curl: (6) Couldn’t resolve host ‘download.fedora.redhat.com’
error: skipping http://download.fedora.redhat.com/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm – transfer failed
[root@tempest ~] rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
Retrieving http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
Preparing…                ########################################### [100%]
package epel-release-6-8.noarch is already installed
[root@tempest ~]

So, he has a package name wrong (the correction was posted as a comment to his page) and we already have both repos installed.  So if installing jpackage-utils doesn’t allow us to find the tomcat6-webapps and tomcat6-manager-webapps, I’ll be very unhappy.

[root@tempest ~] rpm -Uvh http://mirrors.dotsrc.org/jpackage/6.0/generic/free/RPMS/jpackage-utils-5.0.0-7.jpp6.noarch.rpm
Retrieving http://mirrors.dotsrc.org/jpackage/6.0/generic/free/RPMS/jpackage-utils-5.0.0-7.jpp6.noarch.rpm
warning: /var/tmp/rpm-tmp.htRz2i: Header V3 DSA/SHA1 Signature, key ID c431416d: NOKEY
Preparing…                ########################################### [100%]
1:jpackage-utils         warning: /etc/maven/maven2-depmap.xml created as /etc/maven/maven2-depmap.xml.rpmnew
########################################### [100%]
[root@tempest ~]

Okay, at least that added something to our system.  Let’s see how the search goes:

yum –enablerepo=epel,rpmforge search tomcat6 tomcat6-webapps tomcat6-admin-webapps
Loaded plugins: downloadonly, product-id, refresh-packagekit, security, subscription-manager
This system is receiving updates from Red Hat Subscription Management.
rhel-6-server-cf-tools-1-rpms                                                                | 2.8 kB     00:00
rhel-6-server-rhev-agent-rpms                                                                | 3.1 kB     00:00
rhel-6-server-rpms                                                                           | 3.7 kB     00:00
=============================================== N/S Matched: tomcat6 ===============================================
glite-security-trustmanager-tomcat6.noarch : Java trustmanager interface supporting a GSI grid name space
tomcat6.noarch : Apache Servlet/JSP Engine, RI for Servlet 2.5/JSP 2.1 API
tomcat6-el-2.1-api.noarch : Expression Language v1.0 API
tomcat6-jsp-2.1-api.noarch : Apache Tomcat JSP API implementation classes
tomcat6-lib.noarch : Libraries needed to run the Tomcat Web container
tomcat6-servlet-2.5-api.noarch : Apache Tomcat Servlet API implementation classes

Name and summary matches mostly, use “search all” for everything.
Warning: No matches found for: tomcat6-webapps
Warning: No matches found for: tomcat6-admin-webapps
[root@tempest yum.repos.d]

Nope, utter failure.

Hang on, (more googling and, more importantly, logging into my old redhat account and searching the knowledge base finds this page:

https://access.redhat.com/site/solutions/56374

Which promises to exactly answer my question, but since we are now until LTS and I don’t have the password, I can’t read the answer.  So, I asked Andrew Maroney to look at it for me.  Turns out that the two -webapps packages are in RHEL optional, so we have to enable that repo:

[root@tempest yum.repos.d] yum-config-manager –enable rhel-6-server-optional-rpms

and now:

[root@tempest yum.repos.d] yum search tomcat6-webapps tomcat6-admin-webapps
Loaded plugins: downloadonly, product-id, refresh-packagekit, security, subscription-manager
This system is receiving updates from Red Hat Subscription Management.
rhel-6-server-cf-tools-1-rpms                                                                | 2.8 kB     00:00
rhel-6-server-optional-rpms                                                                  | 3.5 kB     00:00
rhel-6-server-rhev-agent-rpms                                                                | 3.1 kB     00:00
rhel-6-server-rpms                                                                           | 3.7 kB     00:00
=========================================== N/S Matched: tomcat6-webapps ===========================================
tomcat6-webapps.noarch : The ROOT and examples web applications for Apache Tomcat

======================================== N/S Matched: tomcat6-admin-webapps ========================================
tomcat6-admin-webapps.noarch : The host-manager and manager web applications for Apache Tomcat

Name and summary matches mostly, use “search all” for everything.
[root@tempest yum.repos.d]

Yay!  So I don’t need EPEL or Rpmforge at all:

[root@tempest yum.repos.d] yum install tomcat6 tomcat6-webapps tomcat6-admin-webapps
Setting up Install Process
Resolving Dependencies
–> Running transaction check
—> Package tomcat6-admin-webapps.noarch 0:6.0.24-62.el6 will be installed
—> Package tomcat6-webapps.noarch 0:6.0.24-62.el6 will be installed
–> Processing Dependency: jakarta-taglibs-standard >= 1.1 for package: tomcat6-webapps-6.0.24-62.el6.noarch
–> Running transaction check
—> Package jakarta-taglibs-standard.noarch 0:1.1.1-11.4.el6 will be installed
—> Package tomcat6.noarch 0:6.0.24-62.el6 will be installed
10gen/filelists                                                                              | 5.3 kB     00:00
rhel-6-server-cf-tools-1-rpms/filelists_db                                                   |  11 kB     00:00
rhel-6-server-optional-rpms/filelists_db                                                     | 4.5 MB     00:01
rhel-6-server-rhev-agent-rpms/filelists_db                                                   |  12 kB     00:00
rhel-6-server-rpms/filelists_db                                                              |  14 MB     00:07
–> Processing Dependency: tomcat6-lib = 6.0.24-62.el6 for package: tomcat6-6.0.24-62.el6.noarch
–> Processing Dependency: jakarta-commons-pool for package: tomcat6-6.0.24-62.el6.noarch
–> Processing Dependency: jakarta-commons-dbcp for package: tomcat6-6.0.24-62.el6.noarch
–> Processing Dependency: jakarta-commons-daemon for package: tomcat6-6.0.24-62.el6.noarch
–> Processing Dependency: jakarta-commons-collections for package: tomcat6-6.0.24-62.el6.noarch
–> Running transaction check
—> Package jakarta-commons-collections.noarch 0:3.2.1-3.4.el6 will be installed
—> Package jakarta-commons-daemon.x86_64 1:1.0.1-8.9.el6 will be installed
—> Package jakarta-commons-dbcp.noarch 0:1.2.1-13.8.el6 will be installed
—> Package jakarta-commons-pool.x86_64 0:1.3-12.7.el6 will be installed
—> Package tomcat6-lib.noarch 0:6.0.24-62.el6 will be installed
–> Processing Dependency: tomcat6-servlet-2.5-api = 6.0.24-62.el6 for package: tomcat6-lib-6.0.24-62.el6.noarch
–> Processing Dependency: tomcat6-jsp-2.1-api = 6.0.24-62.el6 for package: tomcat6-lib-6.0.24-62.el6.noarch
–> Processing Dependency: tomcat6-el-2.1-api = 6.0.24-62.el6 for package: tomcat6-lib-6.0.24-62.el6.noarch
–> Running transaction check
—> Package tomcat6-el-2.1-api.noarch 0:6.0.24-62.el6 will be installed
—> Package tomcat6-jsp-2.1-api.noarch 0:6.0.24-62.el6 will be installed
—> Package tomcat6-servlet-2.5-api.noarch 0:6.0.24-62.el6 will be installed
–> Finished Dependency Resolution

Dependencies Resolved

Installed:
tomcat6-admin-webapps.noarch 0:6.0.24-62.el6                tomcat6-webapps.noarch 0:6.0.24-62.el6

Dependency Installed:
jakarta-commons-collections.noarch 0:3.2.1-3.4.el6         jakarta-commons-daemon.x86_64 1:1.0.1-8.9.el6
jakarta-commons-dbcp.noarch 0:1.2.1-13.8.el6               jakarta-commons-pool.x86_64 0:1.3-12.7.el6
jakarta-taglibs-standard.noarch 0:1.1.1-11.4.el6           tomcat6.noarch 0:6.0.24-62.el6
tomcat6-el-2.1-api.noarch 0:6.0.24-62.el6                  tomcat6-jsp-2.1-api.noarch 0:6.0.24-62.el6
tomcat6-lib.noarch 0:6.0.24-62.el6                         tomcat6-servlet-2.5-api.noarch 0:6.0.24-62.el6

Complete!
[root@tempest yum.repos.d]

Whew!  Okay, let’s try starting tomcat:

[root@tempest yum.repos.d] service tomcat6 start
Starting tomcat6:                                          [  OK  ]

That’s nice.  If we browse to the Tomcat home page, we see the logo; yay!

Now we need to start configuring it.  We should look at the kinds of things we did with Tomcat5 on Puma; hopefully the changes will be similar.

We should also look at these:

http://tomcat.apache.org/tomcat-6.0-doc/setup.html

http://tomcat.apache.org/tomcat-6.0-doc/manager-howto.html

 

 

 

Posted in Uncategorized | Leave a comment

cs111 tutor account

Sohie wants the cs111 tutors to be able to read the contents of /home/cs111/drop so that they can grade dropped assignments, but not to be able to write to other cs111 files and directories. Currently the drop folder is owned by cs111, so there’s no choice but to give the tutors the cs111 password and allow them to write other files.

My idea is to create a cs111tutor account, make its home directory be the cs111/drop folder. It will then have a separate password. If we put the drop folder in the cs111tutor group, the cs111 tutors will also be able to read the files, but won’t own any of them, so they won’t even be able to modify the permissions. The drop folders can still be owned by cs111, but gid of cs111tutor.

One concern is that /home/cs111/drop is a symlink, not a real directory, but that may not matter.

So:

useradd -c "CS111 tutor has access to drop directory" -d /home/cs111/drop -M -U cs111tutor
[root@tempest ~] /usr/bin/passwd cs111tutor
 Changing password for user cs111tutor.
 New password:
 Retype new password:
 passwd: all authentication tokens updated successfully.
 [root@tempest ~] cd ~cs111tutor
[root@tempest drop] ls -ld .
 drwxr-xr-x. 2 cs111 cs111 4096 Feb  2 13:44 .

Now, all the elements of the “drop” directory are symlinks, and we can modify the group of a symlink, but that’s not what we want to do. Instead, we want to modify the group of the thing that the simlink is pointing to:

[root@tempest drop] for a in *; do chgrp -R cs111tutor /students/$a/cs111/; done
[root@tempest drop] ls[root@tempest drop] su - cs111tutor
-bash-4.1$ pwd
/home/cs111/drop/
-bash-4.1$ cd egstu/
-bash-4.1$ ls
exam2  ps01  ps02  ps03  ps04  ps05  ps06  ps07  ps08  ps09  ps10  ps11
-bash-4.1$ ls -l
total 48
dr-xr-s---. 5 egstu cs111tutor 4096 Nov 12 07:20 exam2
drwxr-s---. 4 egstu cs111tutor 4096 Sep  9 17:22 ps01
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps02
drwxr-s---. 3 egstu cs111tutor 4096 Sep 18 18:12 ps03
drwxr-s---. 4 egstu cs111tutor 4096 Sep 30 22:33 ps04
drwxr-s---. 3 egstu cs111tutor 4096 Oct  6 22:33 ps05
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps06
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps07
drwxr-s---. 3 egstu cs111tutor 4096 Oct 30 17:53 ps08
drwxr-s---. 3 egstu cs111tutor 4096 Nov 17 15:37 ps09
drwxr-s---. 4 egstu cs111tutor 4096 Nov 24 11:47 ps10
drwxr-s---. 3 egstu cs111tutor 4096 Dec  4 17:43 ps11
-bash-4.1$logout

We also need to add cs111tutor as a supplementary group for the cs111 user and for Sohie, too:

[root@tempest drop] grep cs111tutor /etc/group
cs111tutor:x:6163:cs111,slee
[root@tempest drop]

If that works, we should be able to read students’ submissions as cs111:

[root@tempest drop] su - cs111
[cs111@tempest ~] cd ~/drop
[cs111@tempest drop] id
uid=709(cs111) gid=709(cs111) groups=709(cs111),3740(cs111web),6163(cs111tutor) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[cs111@tempest drop] cd egstu/
[cs111@tempest egstu] ls -l
total 48
dr-xr-s---. 3 egstu cs111tutor 4096 Nov 11 21:50 exam2
drwxr-s---. 3 egstu cs111tutor 4096 Sep  8 19:27 ps01
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps02
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps03
drwxr-s---. 3 egstu cs111tutor 4096 Sep 30 21:19 ps04
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps05
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps06
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps07
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps08
drwxr-s---. 3 egstu cs111tutor 4096 Nov 18 20:12 ps09
drwxr-s---. 3 egstu cs111tutor 4096 Nov 25 20:54 ps10
drwxr-s---. 2 egstu cs111tutor 4096 Sep  3 21:00 ps11
[cs111@tempest egstu] cd ps01
[cs111@tempest ps01] ls -l
total 4
drwxr-sr-x. 2 egstu cs111tutor 4096 Sep  8 19:27 ps01_programs
[cs111@tempest ps01]

It works!

Posted in Uncategorized | Leave a comment