readme.txt
2022-07-21
looking back, I see what I thought might take a week or at most two, has ended up being over four months of work! The project was to make installing "thttpd" from Acme.com simple enough even a third or fourth grader could build their own World Wide Web Server that is bullet proof! After the process was fairly stable, I added the line number to the end of the comment on the command lines that one would copy and paste into a terminal window. This has proved to be very similar to an "error correction code." Most of the time, my times were way off because I was making changes in the process and this was adding to the time. My last run, as I was hurrying and not paying attention to the computer responses, I missed picking up the "sudo" on the directory copy command and the computer returned "cp: cannot create directory '/home/local/sbin/os-debug': Permission denied." See: "2022-07-12-thttpd-install-test.txt" line 1280, "command ### sudo 1919." Much later in the program I came to the point of having to change to the above directory, and of course it did not exist! Going back in the directions, I found where the directory was supposed to be made by a "sudo copy" so I went back a little farther to make sure I was in the right place (directory) to make the "sudo copy" and then a few steps farther to make sure everything that needed to be done was done. Then it was back to the point where the directory that did not exist, now did exist! and continued to the end with no more problems.
On checking through the process I did find a couple of typos and a coupe of places where I did not leave any space and was able to scrounge some spaces from nearby so I did not have to change a lot of line numbers.
After building thttpd on all of the different Raspberry Pis that I have, since each Pi has a different processor and set of commands and functions built in, I finally saw that there was basically just two different builds, one for the 32 bit OS and one for the 64 bit OS. One can use a Raspberry Pi 400, which is their fastest computer and has 4 GB of RAM, and build the OS and thttpd on the SD card in it. Then one can just install that SD card into the destination Pi.
The Raspberry Pi B Rev 2.0 with 512 MB of RAM, running at 700 MHz with a full size SD card and 2 USB ports did take a full install (2,277 MB) and then I compiled thttpd with no problems. Chromium would not run, or at least it took so long to load I gave up on it. I then installed "ELinks" - A text web browser for computers with little RAM, that runs in a terminal window, that works great with no problems. The browser only needs to be text, since it is only used to see if the web site comes up and if it is actually functioning.
"thttpd" is designed to be as simple as possible, yet conserve to the max on scarce resources.
cd /home/local/sbin ### 1836
frank@raspberrypi:/home/local/sbin $
frank@raspberrypi:/home/local/sbin $ ls -al ### 1841
total 136
drwxr-xr-x 2 root root 4096 Jul 12 14:37 .
drwxr-xr-x 5 root root 4096 Jul 12 14:37 ..
-rwxr-xr-x 1 root root 13692 Jul 12 14:37 htpasswd # 32 bit OS
-rwxr-sr-x 1 root www 12996 Jul 12 14:37 makeweb # 32 bit OS
-r-xr-xr-x 1 root root 2878 Jul 12 14:37 syslogtocern # 32 bit OS
-r-xr-xr-x 1 bin bin 91232 Jul 12 14:37 thttpd # 32 Bit OS.
frank@raspberrypi:/home/local/sbin $
This is for the 64 bit OS on the Raspberry Pi 400 as it now is with os-debug files added:
pi@AC0XL-Pi-400:~/freedom $ cd /home/local/sbin/
pi@AC0XL-Pi-400:/home/local/sbin $ ls -al
total 192
drwxr-xr-x 4 root root 4096 Apr 18 19:42 .
drwxr-xr-x 5 root root 4096 Apr 18 16:39 ..
-rwxr-xr-- 1 root root 185 Apr 18 17:19 datelog
-r-xr-xr-- 1 root root 185 Apr 18 17:19 datelog~
-rwxr-xr-- 1 root root 385 Apr 18 17:19 dead-thttpd
-r-xr-xr-- 1 root root 385 Apr 18 17:19 dead-thttpd~
-rwxr-xr-x 1 root root 15296 Apr 18 16:39 htpasswd # 64 bit OS
-rwxr-sr-x 1 root www 14504 Apr 18 16:39 makeweb # 64 bit OS
drwxr-xr-x 2 root root 4096 Apr 22 15:59 os-debug
-r-xr-xr-x 1 root root 2878 Apr 18 16:39 syslogtocern # 64 bit OS
-r-xr-xr-x 1 bin bin 121272 Apr 18 16:39 thttpd # 64 bit OS
drwxr-xr-x 3 root root 4096 Jun 7 16:40 thttpd-extras
-rwxr-xr-- 1 root root 153 Apr 18 17:19 thttpd_wrapper
pi@AC0XL-Pi-400:/home/local/sbin $
As one can see, the 64 bit OS files are a little bit larger than the 32 bit OS files.
pi@AC0XL-Pi-400:/home/local/sbin $ cd thttpd-extras/
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ ls -al
total 80
drwxr-xr-x 3 root root 4096 Jun 7 16:40 .
drwxr-xr-x 4 root root 4096 Apr 18 19:42 ..
-r--r--r-- 1 root root 533 Apr 18 17:18 anvil_thttpd.gif
-rw-r--r-- 1 root root 533 Apr 18 17:18 favicon.ico
-rwxr-xr-- 1 root root 485 Apr 18 17:18 rc.local
-r-xr-xr-- 1 root root 485 Apr 18 17:18 rc.local~
-rw-r--r-- 1 root root 308 Apr 18 17:18 readme.txt
-r--r--r-- 1 root root 73 Apr 18 17:18 robots-site.txt
-rw-r--r-- 1 root root 27 Apr 18 17:18 robots.txt
-rw-r--r-- 1 root root 186 Apr 18 17:18 sitemap.xml
-r--r--r-- 1 root root 186 Apr 18 17:18 sitemap.xml.txt
-rwxr-xr-- 1 root root 501 Jun 7 16:20 thttpd_config
-r-xr-xr-- 1 root root 501 Jun 7 16:20 thttpd_config~
-rwxr-xr-- 1 root root 1746 Apr 18 17:18 thttpd-rotate
-r-xr-xr-- 1 root root 1746 Apr 18 17:18 thttpd-rotate~
-rwxr-xr-- 1 root root 136 Apr 22 17:00 thttpd.sh
-r-xr-xr-- 1 root root 136 Apr 22 17:02 thttpd.sh~
-rwxr-xr-- 1 root root 122 Apr 18 17:18 thttpd_wrapper
-r-xr-xr-- 1 root root 122 Apr 18 17:18 thttpd_wrapper~
drwxr-xr-x 4 root root 4096 Jun 7 14:50 vhost-DN-test
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $
When the computer comes on the first file that starts thttpd is a line at the end of rc.local (. /etc/thttpd.sh). "rc.local" is a door to "systemd" which is also found in "/etc". Some distributions have dropped "rc.local" I believe because of the line: "By default this script does nothing." These files are in /etc :
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
printf "My IP address is %s\n" "$_IP"
fi
# Start thttpd
echo "Starting thttpd. \n"
. /etc/thttpd.sh
#
exit 0
(. /etc/thttpd.sh) says: do /etc/thttpd.sh, which is the next file to run:
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat thttpd.sh
#!/bin/sh
# thttpd.sh
if [ -x /home/local/sbin/thttpd_wrapper ] ; then
echo " thttpd "
/home/local/sbin/thttpd_wrapper &
fi
#
exit 0
If /home/local/sbin/thttpd_wrapper is executable then
echo " thttpd " and
run /home/local/sbin/thttpd_wrapper in the back ground
end of the if
This takes us to: /home/local/sbin/thttpd_wrapper.
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat thttpd_wrapper
#!/bin/sh
# thttpd_wrapper
while true ; do
/home/local/sbin/thttpd -D -C /home/local/www/thttpd_config
sleep 10
done
#
(/home/local/sbin/thttpd -D)
-D This was originally just a debugging flag, however it's worth
mentioning because one of the things it does is prevent thttpd
from making itself a background daemon. Instead it runs in the
foreground like a regular program. This is necessary when you
want to run thttpd wrapped in a little shell script that
restarts it if it exits.
(-C /home/local/www/thttpd_config)
-C Specifies a config-file to read. All options can be set either
by command-line flags or in the config file.
(sleep 10) go back to sleep for 10 seconds, then check to make sure thttpd is running.
As you can see the thttpd_wrapper makes sure thttpd is running in the foreground and then goes back to sleep so its PID is sleeping almost all of the time, using very little scarce resources!
The config file is likewise very brief:
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat thttpd_config
# /home/local/www/thttpd_config
dir=/home/local/www
chroot #chroot jail
data_dir=users
#vhost # if you use a domain name you will have to uncomment (remove the # at the beginning) and then make a "user" directory using your external IPv4 address (xxx.xxx.xxx.xxx) and inside this directory make symbolic links back up to your other files in "users" (ln -rs ../"user-name"), see: "vhost-DN-test".
cgipat=**.cgi
logfile=/home/local/www/logs/thttpd_log
pidfile=/var/run/thttpd.pid
#
(chroot) says that thttpd can only access files that are in the (dir=/home/local/www) or below, nothing can go higher in the file system. This leaves only access to the config file, the cgi-bin directory, the logs directory, and the users directory.
pi@AC0XL-Pi-400:/home/local/www $ ls -al
total 28
drwxr-xr-x 5 root www 4096 Apr 18 18:07 .
drwxr-xr-x 5 root root 4096 Apr 18 16:39 ..
drwxr-xr-x 2 root www 4096 Apr 18 16:39 cgi-bin
drwxr-xr-x 2 root www 4096 Apr 18 20:16 logs
-rwxr-xr-- 1 root root 501 Jun 7 16:20 thttpd_config
-r-xr-xr-- 1 root root 501 Jun 7 16:20 thttpd_config~
drwxrwxr-x 16 root www 4096 Jun 1 15:24 users
When (vhost) is uncommented, the IP Address for thttpd is added to the beginning of the information that is returned from a web request. This means that the request is looking for a directory that has the IP Address.
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ ls -alR vhost-DN-test/
vhost-DN-test/:
total 16
drwxr-xr-x 4 root root 4096 Jun 7 14:50 .
drwxr-xr-x 3 root root 4096 Jun 7 16:40 ..
drwxr-xr-x 2 root root 4096 Jun 7 16:06 192.168.1.128
drwxr-xr-x 2 root root 4096 Jun 7 14:50 a0
vhost-DN-test/192.168.1.128:
total 12
drwxr-xr-x 2 root root 4096 Jun 7 16:06 .
drwxr-xr-x 4 root root 4096 Jun 7 14:50 ..
lrwxrwxrwx 1 root root 5 Jun 7 14:50 a0 -> ../a0
-rw-r--r-- 1 root root 60 Jun 7 16:04 sample.txt
vhost-DN-test/a0:
total 8
drwxr-xr-x 2 root root 4096 Jun 7 14:50 .
drwxr-xr-x 4 root root 4096 Jun 7 14:50 ..
-rw-r--r-- 1 root root 0 Jun 2 01:45 test
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat vhost-DN-test/192.168.1.128/sample.txt
ln -rs ../a0 ### One may have to use: sudo ln -rs ../a0
Here "a0" is the original user with a directory "test" under it. The new IP Address "192.168.1.128" is added to the users. To this user is a recursive symbolic link back up to the user "a0", and that lets one get to the directory "test" that is under "a0".
The last script is to rotate the log files which are in /etc/cron.daily:
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat thttpd-rotate
#!/bin/sh
# /etc/cron.daily/thttpd-rotate
cd /home/local/www/logs
rm -f thttpd_log.50
mv thttpd_log.49 thttpd_log.50
mv thttpd_log.48 thttpd_log.49
mv thttpd_log.47 thttpd_log.48
mv thttpd_log.46 thttpd_log.47
mv thttpd_log.45 thttpd_log.46
mv thttpd_log.44 thttpd_log.45
mv thttpd_log.43 thttpd_log.44
mv thttpd_log.42 thttpd_log.43
mv thttpd_log.41 thttpd_log.42
mv thttpd_log.40 thttpd_log.41
mv thttpd_log.39 thttpd_log.40
mv thttpd_log.38 thttpd_log.39
mv thttpd_log.37 thttpd_log.38
mv thttpd_log.36 thttpd_log.37
mv thttpd_log.35 thttpd_log.36
mv thttpd_log.34 thttpd_log.35
mv thttpd_log.33 thttpd_log.34
mv thttpd_log.32 thttpd_log.33
mv thttpd_log.31 thttpd_log.32
mv thttpd_log.30 thttpd_log.31
mv thttpd_log.29 thttpd_log.30
mv thttpd_log.28 thttpd_log.29
mv thttpd_log.27 thttpd_log.28
mv thttpd_log.26 thttpd_log.27
mv thttpd_log.25 thttpd_log.26
mv thttpd_log.24 thttpd_log.25
mv thttpd_log.23 thttpd_log.24
mv thttpd_log.22 thttpd_log.23
mv thttpd_log.21 thttpd_log.22
mv thttpd_log.20 thttpd_log.21
mv thttpd_log.19 thttpd_log.20
mv thttpd_log.18 thttpd_log.19
mv thttpd_log.17 thttpd_log.18
mv thttpd_log.16 thttpd_log.17
mv thttpd_log.15 thttpd_log.16
mv thttpd_log.14 thttpd_log.15
mv thttpd_log.13 thttpd_log.14
mv thttpd_log.12 thttpd_log.13
mv thttpd_log.11 thttpd_log.12
mv thttpd_log.10 thttpd_log.11
mv thttpd_log.09 thttpd_log.10
mv thttpd_log.08 thttpd_log.09
mv thttpd_log.07 thttpd_log.08
mv thttpd_log.06 thttpd_log.07
mv thttpd_log.05 thttpd_log.06
mv thttpd_log.04 thttpd_log.05
mv thttpd_log.03 thttpd_log.04
mv thttpd_log.02 thttpd_log.03
mv thttpd_log.01 thttpd_log.02
date >> thttpd_log
w >> thttpd_log
mv thttpd_log thttpd_log.01
date > thttpd_log
w >> thttpd_log
kill -usr1 `cat /var/run/thttpd.pid`
#
Change to /home/local/www/logs
delete log # 50
move all of the logs up one number until log # 0
add the "date" and "w" to the end of log # 0
move log # 0 to log # 1
move "date" to new log # 0
append "w" to log # 0
stop input to thttpd and when stopped kill thttpd
When thttpd_wrapper sees that thttpd is not running it will start it again.
The files that get added to /home/local/www/users:
I liked "anvil_thttpd.gif" since it shows that thttpd can take a beating without failing and copied it to "favicon.ico" which is the icon that appears on the web browser line for the web site.
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat readme.txt
If no index.html type page is found, this web server generates its own index, which is a green screen with the files listed much like a directory page, or an ftp site. All one has to do is click on the file or directory name and it will open! Clicking the two dots (../) will take one back up one level.
*
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat robots.txt
User-agent: *
Disallow:
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat robots-site.txt
User-agent: *
Disallow:
Sitemap: http://xxx.xxx.xxx.xxx/sitemap.xml/
If one is using a static IP Address that will replace the XXX.XXX.XXX.XXX and then save it as: robots.txt
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cat sitemap.xml.txt
http://xxx.xxx.xxx.xxx/
If one is using a static IP Address that will replace the XXX.XXX.XXX.XXX and then save it as: sitemap.xml
The directory os-debug adds PID numbers and lets one also add a date to the log file as a marker. The directions for installing these scripts are in the readme.txt file
pi@AC0XL-Pi-400:/home/local/sbin/thttpd-extras $ cd ../os-debug/
pi@AC0XL-Pi-400:/home/local/sbin/os-debug $ ls -al
total 48
drwxr-xr-x 2 root root 4096 Apr 22 15:59 .
drwxr-xr-x 4 root root 4096 Apr 18 19:42 ..
-rwxr-xr-- 1 root root 185 Apr 18 17:19 datelog
-r-xr-xr-- 1 root root 185 Apr 18 17:19 datelog~
-rwxr-xr-- 1 root root 385 Apr 18 17:19 dead-thttpd
-r-xr-xr-- 1 root root 385 Apr 18 17:19 dead-thttpd~
-r--r--r-- 1 root root 792 Apr 22 17:17 readme.txt
-r--r--r-- 1 root root 792 Apr 22 17:17 readme.txt~
-rwxr-xr-- 1 root root 1965 Apr 18 17:19 thttpd-rotate
-r-xr-xr-- 1 root root 1965 Apr 18 17:19 thttpd-rotate~
-rwxr-xr-- 1 root root 153 Apr 18 17:19 thttpd_wrapper
-r-xr-xr-- 1 root root 153 Apr 18 17:19 thttpd_wrapper~
pi@AC0XL-Pi-400:/home/local/sbin/os-debug $ cat readme.txt
#readme.txt
#
# The files in os-debug have additional thttpd pid numbers to help with OS problems.
#
#pi@AC0XL-Pi-400:~$
#pi@AC0XL-Pi-400:~$ cd /home/local/sbin/os-debug/ ###
#pi@AC0XL-Pi-400:/home/local/sbin/os-debug $
#pi@AC0XL-Pi-400:/home/local/sbin/os-debug $ sudo cp -av d* thttp*r ../ ### sudo
#'datelog' -> '../datelog'
#'dead-thttpd' -> '../dead-thttpd'
#'thttpd_wrapper' -> '../thttpd_wrapper'
#pi@AC0XL-Pi-400:/home/local/sbin/os-debug $
#pi@AC0XL-Pi-400:/home/local/sbin/os-debug $
#
# Now we have to replace thttpd-rotate in /etc/cron.daily.
#
#pi@AC0XL-Pi-400:/home/local/sbin/os-debug $ sudo cp -av thttpd-rotate /etc/cron.daily ### sudo - we are adding lines for thttpd pid information.
#pi@AC0XL-Pi-400:/home/local/sbin/os-debug $
#
pi@AC0XL-Pi-400:/home/local/sbin/os-debug $ cat datelog
#!/bin/sh
# datelog
echo "datelog, thttpd pid = " `ps -C thttpd` >> /home/local/www/logs/thttpd_log
date >> /home/local/www/logs/thttpd_log
w >> /home/local/www/logs/thttpd_log
#
pi@AC0XL-Pi-400:/home/local/sbin/os-debug $ cat dead-thttpd
#!/bin/sh
# dead-thttpd
echo "old dead-thttpd pid = " `cat /var/run/thttpd.pid` >> /home/local/www/logs/thttpd_log
date >> /home/local/www/logs/thttpd_log
w >> /home/local/www/logs/thttpd_log
# sleep 15
# echo "dead-thttpd, new thttpd pid = " `ps -C thttpd` >> /home/local/www/logs/thttpd_log
# date >> /home/local/www/logs/thttpd_log
# w >> /home/local/www/logs/thttpd_log
#
pi@AC0XL-Pi-400:/home/local/sbin/os-debug $ cat thttpd-rotate
#!/bin/sh
# /etc/cron.daily/thttpd-rotate
cd /home/local/www/logs
rm -f thttpd_log.50
mv thttpd_log.49 thttpd_log.50
mv thttpd_log.48 thttpd_log.49
mv thttpd_log.47 thttpd_log.48
mv thttpd_log.46 thttpd_log.47
mv thttpd_log.45 thttpd_log.46
mv thttpd_log.44 thttpd_log.45
mv thttpd_log.43 thttpd_log.44
mv thttpd_log.42 thttpd_log.43
mv thttpd_log.41 thttpd_log.42
mv thttpd_log.40 thttpd_log.41
mv thttpd_log.39 thttpd_log.40
mv thttpd_log.38 thttpd_log.39
mv thttpd_log.37 thttpd_log.38
mv thttpd_log.36 thttpd_log.37
mv thttpd_log.35 thttpd_log.36
mv thttpd_log.34 thttpd_log.35
mv thttpd_log.33 thttpd_log.34
mv thttpd_log.32 thttpd_log.33
mv thttpd_log.31 thttpd_log.32
mv thttpd_log.30 thttpd_log.31
mv thttpd_log.29 thttpd_log.30
mv thttpd_log.28 thttpd_log.29
mv thttpd_log.27 thttpd_log.28
mv thttpd_log.26 thttpd_log.27
mv thttpd_log.25 thttpd_log.26
mv thttpd_log.24 thttpd_log.25
mv thttpd_log.23 thttpd_log.24
mv thttpd_log.22 thttpd_log.23
mv thttpd_log.21 thttpd_log.22
mv thttpd_log.20 thttpd_log.21
mv thttpd_log.19 thttpd_log.20
mv thttpd_log.18 thttpd_log.19
mv thttpd_log.17 thttpd_log.18
mv thttpd_log.16 thttpd_log.17
mv thttpd_log.15 thttpd_log.16
mv thttpd_log.14 thttpd_log.15
mv thttpd_log.13 thttpd_log.14
mv thttpd_log.12 thttpd_log.13
mv thttpd_log.11 thttpd_log.12
mv thttpd_log.10 thttpd_log.11
mv thttpd_log.09 thttpd_log.10
mv thttpd_log.08 thttpd_log.09
mv thttpd_log.07 thttpd_log.08
mv thttpd_log.06 thttpd_log.07
mv thttpd_log.05 thttpd_log.06
mv thttpd_log.04 thttpd_log.05
mv thttpd_log.03 thttpd_log.04
mv thttpd_log.02 thttpd_log.03
mv thttpd_log.01 thttpd_log.02
date >> thttpd_log
echo "log rotate end, thttpd pid = " `ps -C thttpd` >> thttpd_log
w >> thttpd_log
mv thttpd_log thttpd_log.01
date > thttpd_log
echo "old log rotate end, old thttpd pid = " `ps -C thttpd` >> thttpd_log
kill -usr1 `cat /var/run/thttpd.pid`
sleep 15
echo "log rotate new, thttpd pid = " `ps -C thttpd` >> thttpd_log
w >> thttpd_log
#
This adds PID numbers and then waits for 15 seconds for the new PID number and then adds it.
pi@AC0XL-Pi-400:/home/local/sbin/os-debug $ cat thttpd_wrapper
#!/bin/sh
# thttpd_wrapper
while true ; do
/home/local/sbin/thttpd -D -C /home/local/www/thttpd_config
/home/local/sbin/dead-thttpd
sleep 10
done
#
This adds "/home/local/sbin/dead-thttpd" to show the old PID number.
When os-debug is used these file replace the files that were in thttpd-extras.
As one can see the files for thttpd are rather minimal and very well suited for situations where scarce resources are at a premium! When one reads through the thttpd man page they will see that thttpd is much more than just a minimal system.
pi@AC0XL-Pi-400:~/freedom $
pi@AC0XL-Pi-400:~/freedom $ cat thttpd-man-page.txt
thttpd(8) System Manager's Manual thttpd(8)
NAME
thttpd - tiny/turbo/throttling HTTP server
SYNOPSIS
thttpd [-C configfile] [-p port] [-d dir] [-dd data_dir] [-r|-nor]
[-s|-nos] [-v|-nov] [-g|-nog] [-u user] [-c cgipat] [-t throttles] [-h
host] [-l logfile] [-i pidfile] [-T charset] [-P P3P] [-M maxage] [-V]
[-D]
DESCRIPTION
thttpd is a simple, small, fast, and secure HTTP server. It doesn't
have a lot of special features, but it suffices for most uses of the
web, it's about as fast as the best full-featured servers (Apache,
NCSA, Netscape), and it has one extremely useful feature (URL-traffic-
based throttling) that no other server currently has.
OPTIONS
-C Specifies a config-file to read. All options can be set either
by command-line flags or in the config file. See below for de‐
tails.
-p Specifies an alternate port number to listen on. The default is
80. The config-file option name for this flag is "port", and
the config.h option is DEFAULT_PORT.
-d Specifies a directory to chdir() to at startup. This is merely
a convenience - you could just as easily do a cd in the shell
script that invokes the program. The config-file option name
for this flag is "dir", and the config.h options are WEBDIR,
USE_USER_DIR.
-r Do a chroot() at initialization time, restricting file access to
the program's current directory. If -r is the compiled-in de‐
fault, then -nor disables it. See below for details. The con‐
fig-file option names for this flag are "chroot" and "nochroot",
and the config.h option is ALWAYS_CHROOT.
-dd Specifies a directory to chdir() to after chrooting. If you're
not chrooting, you might as well do a single chdir() with the -d
flag. If you are chrooting, this lets you put the web files in
a subdirectory of the chroot tree, instead of in the top level
mixed in with the chroot files. The config-file option name for
this flag is "data_dir".
-nos Don't do explicit symbolic link checking. Normally, thttpd ex‐
plicitly expands any symbolic links in filenames, to check that
the resulting path stays within the original document tree. If
you want to turn off this check and save some CPU time, you can
use the -nos flag, however this is not recommended. Note,
though, that if you are using the chroot option, the symlink
checking is unnecessary and is turned off, so the safe way to
save those CPU cycles is to use chroot. The config-file option
names for this flag are "symlinkcheck" and "nosymlinkcheck".
-v Do el-cheapo virtual hosting. If -v is the compiled-in default,
then -nov disables it. See below for details. The config-file
option names for this flag are "vhost" and "novhost", and the
config.h option is ALWAYS_VHOST.
-g Use a global passwd file. This means that every file in the en‐
tire document tree is protected by the single .htpasswd file at
the top of the tree. Otherwise the semantics of the .htpasswd
file are the same. If this option is set but there is no .ht‐
passwd file in the top-level directory, then thttpd proceeds as
if the option was not set - first looking for a local .htpasswd
file, and if that doesn't exist either then serving the file
without any password. If -g is the compiled-in default, then
-nog disables it. The config-file option names for this flag
are "globalpasswd" and "noglobalpasswd", and the config.h option
is ALWAYS_GLOBAL_PASSWD.
-u Specifies what user to switch to after initialization when
started as root. The default is "nobody". The config-file op‐
tion name for this flag is "user", and the config.h option is
DEFAULT_USER.
-c Specifies a wildcard pattern for CGI programs, for instance
"**.cgi" or "/cgi-bin/*". See below for details. The config-
file option name for this flag is "cgipat", and the config.h op‐
tion is CGI_PATTERN.
-t Specifies a file of throttle settings. See below for details.
The config-file option name for this flag is "throttles".
-h Specifies a hostname to bind to, for multihoming. The default
is to bind to all hostnames supported on the local machine. See
below for details. The config-file option name for this flag is
"host", and the config.h option is SERVER_NAME.
-l Specifies a file for logging. If no -l argument is specified,
thttpd logs via syslog(). If "-l /dev/null" is specified,
thttpd doesn't log at all. The config-file option name for this
flag is "logfile".
-i Specifies a file to write the process-id to. If no file is
specified, no process-id is written. You can use this file to
send signals to thttpd. See below for details. The config-file
option name for this flag is "pidfile".
-T Specifies the character set to use with text MIME types. The
default is UTF-8. The config-file option name for this flag is
"charset", and the config.h option is DEFAULT_CHARSET.
-P Specifies a P3P server privacy header to be returned with all
responses. See http://www.w3.org/P3P/ for details. Thttpd
doesn't do anything at all with the string except put it in the
P3P: response header. The config-file option name for this flag
is "p3p".
-M Specifies the number of seconds to be used in a "Cache-Control:
max-age" header to be returned with all responses. An equiva‐
lent "Expires" header is also generated. The default is no
Cache-Control or Expires headers, which is just fine for most
sites. The config-file option name for this flag is "max_age".
-V Shows the current version info.
-D This was originally just a debugging flag, however it's worth
mentioning because one of the things it does is prevent thttpd
from making itself a background daemon. Instead it runs in the
foreground like a regular program. This is necessary when you
want to run thttpd wrapped in a little shell script that
restarts it if it exits.
CONFIG-FILE
All the command-line options can also be set in a config file. One ad‐
vantage of using a config file is that the file can be changed, and
thttpd will pick up the changes with a restart.
The syntax of the config file is simple, a series of "option" or "op‐
tion=value" separated by whitespace. The option names are listed above
with their corresponding command-line flags.
CHROOT
chroot() is a system call that restricts the program's view of the
filesystem to the current directory and directories below it. It be‐
comes impossible for remote users to access any file outside of the
initial directory. The restriction is inherited by child processes, so
CGI programs get it too. This is a very strong security measure, and
is recommended. The only downside is that only root can call chroot(),
so this means the program must be started as root. However, the last
thing it does during initialization is to give up root access by becom‐
ing another user, so this is safe.
The program can also be compile-time configured to always do a ch‐
root(), without needing the -r flag.
Note that with some other web servers, such as NCSA httpd, setting up a
directory tree for use with chroot() is complicated, involving creating
a bunch of special directories and copying in various files. With
thttpd it's a lot easier, all you have to do is make sure any shells,
utilities, and config files used by your CGI programs and scripts are
available. If you have CGI disabled, or if you make a policy that all
CGI programs must be written in a compiled language such as C and stat‐
ically linked, then you probably don't have to do any setup at all.
However, one thing you should do is tell syslogd about the chroot tree,
so that thttpd can still generate syslog messages. Check your system's
syslogd man page for how to do this. In FreeBSD you would put some‐
thing like this in /etc/rc.conf:
syslogd_flags="-l /usr/local/www/data/dev/log"
Substitute in your own chroot tree's pathname, of course. Don't worry
about creating the log socket, syslogd wants to do that itself. (You
may need to create the dev directory.) In Linux the flag is -a instead
of -l, and there may be other differences.
Relevant config.h option: ALWAYS_CHROOT.
CGI
thttpd supports the CGI 1.1 spec.
In order for a CGI program to be run, its name must match the pattern
specified either at compile time or on the command line with the -c
flag. This is a simple shell-style filename pattern. You can use * to
match any string not including a slash, or ** to match any string in‐
cluding slashes, or ? to match any single character. You can also use
multiple such patterns separated by |. The patterns get checked
against the filename part of the incoming URL. Don't forget to quote
any wildcard characters so that the shell doesn't mess with them.
Restricting CGI programs to a single directory lets the site adminis‐
trator review them for security holes, and is strongly recommended. If
there are individual users that you trust, you can enable their direc‐
tories too.
If no CGI pattern is specified, neither here nor at compile time, then
CGI programs cannot be run at all. If you want to disable CGI as a se‐
curity measure, that's how you do it, just comment out the patterns in
the config file and don't run with the -c flag.
Note: the current working directory when a CGI program gets run is the
directory that the CGI program lives in. This isn't in the CGI 1.1
spec, but it's what most other HTTP servers do.
Relevant config.h options: CGI_PATTERN, CGI_TIMELIMIT, CGI_NICE,
CGI_PATH, CGI_LD_LIBRARY_PATH, CGIBINDIR.
BASIC AUTHENTICATION
Basic Authentication is available as an option at compile time. If en‐
abled, it uses a password file in the directory to be protected, called
.htpasswd by default. This file is formatted as the familiar colon-
separated username/encrypted-password pair, records delimited by new‐
lines. The protection does not carry over to subdirectories. The
utility program htpasswd(1) is included to help create and modify .ht‐
passwd files.
Relevant config.h option: AUTH_FILE
THROTTLING
The throttle file lets you set maximum byte rates on URLs or URL
groups. You can optionally set a minimum rate too. The format of the
throttle file is very simple. A # starts a comment, and the rest of
the line is ignored. Blank lines are ignored. The rest of the lines
should consist of a pattern, whitespace, and a number. The pattern is
a simple shell-style filename pattern, using ?/**/*, or multiple such
patterns separated by |.
The numbers in the file are byte rates, specified in units of bytes per
second. For comparison, a v.90 modem gives about 5000 B/s depending on
compression, a double-B-channel ISDN line about 12800 B/s, and a T1
line is about 150000 B/s. If you want to set a minimum rate as well,
use number-number.
Example:
# throttle file for www.acme.com
** 2000-100000 # limit total web usage to 2/3 of our T1,
# but never go below 2000 B/s
**.jpg|**.gif 50000 # limit images to 1/3 of our T1
**.mpg 20000 # and movies to even less
jef/** 20000 # jef's pages are too popular
Throttling is implemented by checking each incoming URL filename
against all of the patterns in the throttle file. The server accumu‐
lates statistics on how much bandwidth each pattern has accounted for
recently (via a rolling average). If a URL matches a pattern that has
been exceeding its specified limit, then the data returned is actually
slowed down, with pauses between each block. If that's not possible
(e.g. for CGI programs) or if the bandwidth has gotten way larger than
the limit, then the server returns a special code saying 'try again
later'.
The minimum rates are implemented similarly. If too many people are
trying to fetch something at the same time, throttling may slow down
each connection so much that it's not really useable. Furthermore, all
those slow connections clog up the server, using up file handles and
connection slots. Setting a minimum rate says that past a certain
point you should not even bother - the server returns the 'try again
later" code and the connection isn't even started.
There is no provision for setting a maximum connections/second throt‐
tle, because throttling a request uses as much cpu as handling it, so
there would be no point. There is also no provision for throttling the
number of simultaneous connections on a per-URL basis. However you can
control the overall number of connections for the whole server very
simply, by setting the operating system's per-process file descriptor
limit before starting thttpd. Be sure to set the hard limit, not the
soft limit.
MULTIHOMING
Multihoming means using one machine to serve multiple hostnames. For
instance, if you're an internet provider and you want to let all of
your customers have customized web addresses, you might have
www.joe.acme.com, www.jane.acme.com, and your own www.acme.com, all
running on the same physical hardware. This feature is also known as
"virtual hosts". There are three steps to setting this up.
One, make DNS entries for all of the hostnames. The current way to do
this, allowed by HTTP/1.1, is to use CNAME aliases, like so:
www.acme.com IN A 192.100.66.1
www.joe.acme.com IN CNAME www.acme.com
www.jane.acme.com IN CNAME www.acme.com
However, this is incompatible with older HTTP/1.0 browsers. If you
want to stay compatible, there's a different way - use A records in‐
stead, each with a different IP address, like so:
www.acme.com IN A 192.100.66.1
www.joe.acme.com IN A 192.100.66.200
www.jane.acme.com IN A 192.100.66.201
This is bad because it uses extra IP addresses, a somewhat scarce re‐
source. But if you want people with older browsers to be able to visit
your sites, you still have to do it this way.
Step two. If you're using the modern CNAME method of multihoming, then
you can skip this step. Otherwise, using the older multiple-IP-address
method you must set up IP aliases or multiple interfaces for the extra
addresses. You can use ifconfig(8)'s alias command to tell the machine
to answer to all of the different IP addresses. Example:
ifconfig le0 www.acme.com
ifconfig le0 www.joe.acme.com alias
ifconfig le0 www.jane.acme.com alias
If your OS's version of ifconfig doesn't have an alias command, you're
probably out of luck (but see http://www.acme.com/soft‐
ware/thttpd/notes.html).
Third and last, you must set up thttpd to handle the multiple hosts.
The easiest way is with the -v flag, or the ALWAYS_VHOST config.h op‐
tion. This works with either CNAME multihosting or multiple-IP multi‐
hosting. What it does is send each incoming request to a subdirectory
based on the hostname it's intended for. All you have to do in order
to set things up is to create those subdirectories in the directory
where thttpd will run. With the example above, you'd do like so:
mkdir www.acme.com www.joe.acme.com www.jane.acme.com
If you're using old-style multiple-IP multihosting, you should also
create symbolic links from the numeric addresses to the names, like so:
ln -s www.acme.com 192.100.66.1
ln -s www.joe.acme.com 192.100.66.200
ln -s www.jane.acme.com 192.100.66.201
This lets the older HTTP/1.0 browsers find the right subdirectory.
There's an optional alternate step three if you're using multiple-IP
multihosting: run a separate thttpd process for each hostname, using
the -h flag to specify which one is which. This gives you more flexi‐
bility, since you can run each of these processes in separate directo‐
ries, with different throttle files, etc. Example:
thttpd -r -d /usr/www -h www.acme.com
thttpd -r -d /usr/www/joe -u joe -h www.joe.acme.com
thttpd -r -d /usr/www/jane -u jane -h www.jane.acme.com
But remember, this multiple-process method does not work with CNAME
multihosting - for that, you must use a single thttpd process with the
-v flag.
CUSTOM ERRORS
thttpd lets you define your own custom error pages for the various HTTP
errors. There's a separate file for each error number, all stored in
one special directory. The directory name is "errors", at the top of
the web directory tree. The error files should be named "errNNN.html",
where NNN is the error number. So for example, to make a custom error
page for the authentication failure error, which is number 401, you
would put your HTML into the file "errors/err401.html". If no custom
error file is found for a given error number, then the usual built-in
error page is generated.
If you're using the virtual hosts option, you can also have different
custom error pages for each different virtual host. In this case you
put another "errors" directory in the top of that virtual host's web
tree. thttpd will look first in the virtual host errors directory, and
then in the server-wide errors directory, and if neither of those has
an appropriate error file then it will generate the built-in error.
NON-LOCAL REFERRERS
Sometimes another site on the net will embed your image files in their
HTML files, which basically means they're stealing your bandwidth. You
can prevent them from doing this by using non-local referrer filtering.
With this option, certain files can only be fetched via a local refer‐
rer. The files have to be referenced by a local web page. If a web
page on some other site references the files, that fetch will be
blocked. There are three config-file variables for this feature:
urlpat A wildcard pattern for the URLs that should require a local re‐
ferrer. This is typically just image files, sound files, and so
on. For example:
urlpat=**.jpg|**.gif|**.au|**.wav
For most sites, that one setting is all you need to enable re‐
ferrer filtering.
noemptyreferrers
By default, requests with no referrer at all, or a null refer‐
rer, or a referrer with no apparent hostname, are allowed. With
this variable set, such requests are disallowed.
localpat
A wildcard pattern that specifies the local host or hosts. This
is used to determine if the host in the referrer is local or
not. If not specified it defaults to the actual local hostname.
SYMLINKS
thttpd is very picky about symbolic links. Before delivering any file,
it first checks each element in the path to see if it's a symbolic
link, and expands them all out to get the final actual filename. Along
the way it checks for things like links with ".." that go above the
server's directory, and absolute symlinks (ones that start with a /).
These are prohibited as security holes, so the server returns an error
page for them. This means you can't set up your web directory with a
bunch of symlinks pointing to individual users' home web directories.
Instead you do it the other way around - the user web directories are
real subdirs of the main web directory, and in each user's home dir
there's a symlink pointing to their actual web dir.
The CGI pattern is also affected - it gets matched against the fully-
expanded filename. So, if you have a single CGI directory but then put
a symbolic link in it pointing somewhere else, that won't work. The
CGI program will be treated as a regular file and returned to the
client, instead of getting run. This could be confusing.
PERMISSIONS
thttpd is also picky about file permissions. It wants data files
(HTML, images) to be world readable. Readable by the group that the
thttpd process runs as is not enough - thttpd checks explicitly for the
world-readable bit. This is so that no one ever gets surprised by a
file that's not set world-readable and yet somehow is readable by the
HTTP server and therefore the *whole* world.
The same logic applies to directories. As with the standard Unix "ls"
program, thttpd will only let you look at the contents of a directory
if its read bit is on; but as with data files, this must be the world-
read bit, not just the group-read bit.
thttpd also wants the execute bit to be *off* for data files. A file
that is marked executable but doesn't match the CGI pattern might be a
script or program that got accidentally left in the wrong directory.
Allowing people to fetch the contents of the file might be a security
breach, so this is prohibited. Of course if an executable file *does*
match the CGI pattern, then it just gets run as a CGI.
In summary, data files should be mode 644 (rw-r--r--), directories
should be 755 (rwxr-xr-x) if you want to allow indexing and 711
(rwx--x--x) to disallow it, and CGI programs should be mode 755 (rwxr-
xr-x) or 711 (rwx--x--x).
LOGS
thttpd does all of its logging via syslog(3). The facility it uses is
configurable. Aside from error messages, there are only a few log en‐
try types of interest, all fairly similar to CERN Common Log Format:
Aug 6 15:40:34 acme thttpd[583]: 165.113.207.103 - - "GET /file" 200 357
Aug 6 15:40:43 acme thttpd[583]: 165.113.207.103 - - "HEAD /file" 200 0
Aug 6 15:41:16 acme thttpd[583]: referrer http://www.acme.com/ -> /dir
Aug 6 15:41:16 acme thttpd[583]: user-agent Mozilla/1.1N
The package includes a script for translating these log entries info
CERN-compatible files. Note that thttpd does not translate numeric IP
addresses into domain names. This is both to save time and as a minor
security measure (the numeric address is harder to spoof).
Relevant config.h option: LOG_FACILITY.
If you'd rather log directly to a file, you can use the -l command-line
flag. But note that error messages still go to syslog.
SIGNALS
thttpd handles a couple of signals, which you can send via the standard
Unix kill(1) command:
INT,TERM
These signals tell thttpd to shut down immediately. Any re‐
quests in progress get aborted.
USR1 This signal tells thttpd to shut down as soon as it's done ser‐
vicing all current requests. In addition, the network socket it
uses to accept new connections gets closed immediately, which
means a fresh thttpd can be started up immediately.
USR2 This signal tells thttpd to generate the statistics syslog mes‐
sages immediately, instead of waiting for the regular hourly up‐
date.
HUP This signal tells thttpd to close and re-open its (non-syslog)
log file, for instance if you rotated the logs and want it to
start using the new one. This is a little tricky to set up cor‐
rectly, for instance if you are using chroot() then the log file
must be within the chroot tree, but it's definitely doable.
SEE ALSO
redirect(8), ssi(8), makeweb(1), htpasswd(1), syslogtocern(8), web‐
log_parse(1), http_get(1)
THANKS
Many thanks to contributors, reviewers, testers: John LoVerso, Jordan
Hayes, Chris Torek, Jim Thompson, Barton Schaffer, Geoff Adams, Dan
Kegel, John Hascall, Bennett Todd, KIKUCHI Takahiro, Catalin Ionescu.
Special thanks to Craig Leres for substantial debugging and develop‐
ment, and for not complaining about my coding style very much.
AUTHOR
Copyright © 1995,1998,1999,2000 by Jef Poskanzer .
All rights reserved.
29 February 2000 thttpd(8)
pi@AC0XL-Pi-400:~/freedom $
pi@AC0XL-Pi-400:~/freedom $
"thttpd" is designed to be as simple as possible, yet conserve to the max on scarce resources. thttpd uses only two PIDs. One for thttpd_wrapper which checks to see if thttpd is running and then sleeps for ten seconds and repeats. The second is for the binary thttpd program which is 91,232 bytes long for the 32 bit OS version, or, 121,272 bytes long for the 64 bin OS version.
When I ran a couple of tests using Apache2 the "wget" rates were a little higher. Apache2 was using three PIDs, one I think for control, and two each running pre-start pools of processes handling multiple requests. Again, not a real good choice for handling scarce resources The binaries for Apache2 are much larger and are scattered over many places. The main binary is in "/usr" the log file is in "/var" and all of the modules are who knows where.
I spent over four hours trying to find where the log file was and finally found it under "/var". out of the box the log file is set to be deleted after 120 minutes. This might be great for an in home situation, but is worthless in a World Wide Web situation. In a few days, I will have over three years of logs! They show what is being looked at, links that do not work, how fast your pages are loading, how many accesses it takes to get a page to load, and the speed of those accesses.
One of the pages I have that has 78 small jpgs that one can click on and the larger jpeg will come up is, "Some Arch Photos from Stan Jones; Page, Arizona." (ac0xl/www/2001-ArchHunterBooks/link.html (Contacts)), and also (ac0xl/www/2002-ArchHunterBooks.html) then the link on the page. As I remember, someone from Germany, half-way around the world, clicked on the link and in two or three seconds, all of the files were fetched and delivered. My internet uplink speed is throttled to only 1 Mbps or 125 Kbps by my provider.
I have found some of the worst pages are made with WordPress. It might be a great and easy program to use to prototype a web page design, but then it needs to be re-written in HTML so it does not take forever to load, and as it loads, keep jerking back to the top of the page. Design for Function over Glitz!
2022-05-19
The files for this update are in: "2022-05-03-AC0XL-Pi-B-Plus-1-2".
For a couple of months I have been working on trying to get the "thttpd" install easier for 3rd or 4th graders to be able to make their own World Wide Web server. After I finished the "2022-04-18-AC0XL-Pi-400-thttpd-install.txt" I thought of trying some smaller versions, but all of the new Raspberry Pi's are out of stock. So I dug through some of my boxes and found a couple of older Pi's. One was a Raspberry Pi B Rev 2.0 with 512 MB RAM that runs at 700 MHz (697.95 BogoMIPS), with two USB ports and a full sized SD card slot. This one ran pretty hot, but it probably did not help that it was about 98 degrees F. inside my trailer. The specs show that model does use a lot more power, thus the extra heat.
Another one I found is the Raspberry Pi Model B Plus Rev 1.2 with 512 MB RAM running at 700 MHz (697.96 BogoMIPS), and the same processor. It has four USB ports and a microSD card with a push-push slot. This one ran cool even at the same inside temperature. The question was, "Will this pi run "thttpd" since the "Chromium Web Browser" was VERY SLOW! running with the new Raspberry Pi OS.
I actually compiled and ran "thttpd" on the Pi B Plus, but when it came time to test to see if thttpd was running, Chromium just did not want to start, so I installed ELinks so I could see if thttpd was showing up on "localhost". Actually the first time I ended up just using my Android phone and typing in the IP Address and it was there!
"2022-04-30-AC0XL-Pi-B-Plus-install.txt" is a copy of the second install, to have a record of how long it takes to do the install and tests and then install my 42 GB of online web data on the Pi B Plus (AC0XL-Pi-B-Plus-1-2-web-size.txt -- 44,016,524,750 bytes in /home/local/www/users/) and then do a "wget" and see how long it takes to copy the web site to a 1 TB hard drive on my Pi 400 using ethernet cables to my gigabit router that also has my web site running and some other android devices and an ONN Roku TV (with 3 HDMI ports) for the Pi B Plus monitor.
AC0XL-Pi-B-Plus-1-2-web-size.txt -- 44,016,524,750 bytes in /home/local/www/users/, the first test 05/03/2022 ran 8638 seconds = 5,095,685 bytes/second average served from the Raspberry Pi B Plus version 1.2 running thttpd as a World Wide Web server! The second test 05/07/2022 I added "--http-keep-alive" to "wget" which I believe could have caused it to run a little slower even though it is the built in default. The second test ended up with 44,016,524,750 bytes in 8911 seconds = 4,939,572 bytes/second average. My internet speed is 2Mb/sec down and 1Mb/sec up which would be 250,000 bytes/sec down and 125,000 bytes/sec up. This is roughly 40 times faster than my internet upload speed! from a lowly Raspberry Pi B Plus!!!
The third test I ran was from my Raspberry Pi 4B which is actively serving with AC0XL-www-Pi-4B-2022-05-14-web-size.txt showing 44,023,600,182 bytes in /home/local/www/users/. AC0XL-www-Pi-4B-2022.05.14-log shows wget started at: "192.168.1.187 - - [13/May/2022:15:10:00 +0000] "GET / HTTP/1.1" 200 25000 "" "Wget/1.21"", and ended: "192.168.1.187 - - [13/May/2022:15:32:59 +0000] "GET /ac0xl/www/2009-GreenRiver.UT/greenriverutah.com/archives/pics/album/res/javascript.js HTTP/1.1" 403 0 "http://192.168.1.2/ac0xl/www/2009-GreenRiver.UT/greenriverutah.com/archives/pics/album/index.html" "Wget/1.21"". So 23 minutes times 60 seconds = 1380 seconds. 44.023,600.182 bytes in 1380 seconds = 31,901,160 bytes/second, from a Pi 4B Gigabit ethernet port to a Gigabit router that is also serving my live WWW connection, then to a Pi 400 Gigabit ethernet port. I did see a few connections made from the outside during the wget. This is about 255.20928 times faster than my upload speed!!!
Think of "thttpd" as the bullet proof skeleton web server, raw data (files which are read-only) are the meat on the bones, and HTML is the skin covering the meat, which does contain "POST" and might open a vulnerability since it is a write, with CGI being the bells and whistles, which contain programs that could easily contain vulnerabilities. Out of the box "thttpd" is safe because it is read-only, whereas a "LAMP" install (Linux, Apache, MySQL, and PHP) out of the box might last a few days, if not just a few hours when exposed to the World Wide Web and the attacks that are all of the time happening!
*************************************************
CGI
Author
Vangie Beal
September 1, 1996
Updated on: February 10, 2022
Common Gateway Interface
Common Gateway Interface is an interface specification for transferring information between WWW servers and external databases and information sources known as CGI programs (sometimes referred to as scripts). The specifics of how the script is executed is determined by the server. A CGI program is any program designed to accept and return data that conforms to the CGI’s specification and is the most common way for web servers to interact dynamically with users.
History of Common Gateway Interface specifications
In 1993, The National Center for Supercomputing Applications (NCSA) developed standards in 1993 to interface with servers, like HTTP, to execute any external program, and developers accepted it as a standard for web servers. In 1997 NCSA formed a team—including Rob McCool, the author of NCSA HTTPd Web Server, and Ken Coar—and they formally defined CGI, which is specified in RFC 3875: “The Common Gateway Interface, or CGI, is a standard for external gateway programs to interface with information servers such as HTTP servers.”
Common Gateway Interface features
The following are some of the most significant features of CGI that attracts developers to generate dynamic web content:
Highly compatible with all web browsers.
Written in a simple and clear scripting language like Perl or C.
Can easily interface with HTML.
For example, when a user fills out a form on a Web page and submits it, it needs to be processed by an application program. The Web server passes the form information to a small application program that then processes the data and sends a confirmation message back. This passing of data back and forth between the server and application is the CGI; it works as a mechanism that is part of the Hypertext Transfer Protocol (HTTP).
Common Gateway Interface advantages
Being a powerful data exchanging mechanism between the servers and the external databases, CGI web technology possesses the following advantages to users:
Users can perform advanced tasks much easier by using CGI instead of using other programming languages like Java.
It is easy to use an existing code rather than writing a new code from scratch.
CGI programs are language-independent; therefore, they can be written in any language.
Counters and codes based on CGI are available to perform simple tasks.
Common Gateway Interface disadvantages
Although CGI provides enough flexibility and versatility to run applications on a server, it has some disadvantages:
Page loading incurs overhead, as it needs to call on a new process each time an HTTP request is initiated, and it uses significant server memory. Therefore, CGI scripts are not suitable for high-traffic web pages.
The process of data caching between page loads is normally slow under CGI.
Though CGI has a huge codebase, it is mostly in Perl and is not suitable for developing client-side applications.
Scripts take a lot of processing time, as they need to create a separate process each time an HTTP request is initiated.
Another disadvantage is that a new process is started each time a script is executed. For busy websites, this can noticeably slow down the server. Using the server’s API, such as ISAPI or NSAPI, can be more efficient but difficult to implement. Another popular solution is using Java’s servlets.
Common Gateway Interface alternatives
Due to some disadvantages like processing time and slow data caching processes, the use of CGI is falling. Here are some alternatives to CGI:
ASP
Active Server Pages (ASP) is a web technology based on Internet Information Services (IIS) developed by Microsoft to develop dynamic web pages and quickly respond to the request initiated from HTML forms. ASP scripts are performed on the server, and it also contains HTML, XML, and text. While compared with CGI, ASP is simple, secure, and fast.
PHP
Hypertext Preprocessor, popularly known as PHP, is a server-side open-source scripting language. Like ASP and CGI, PHP scripts are performed on the server, and it comprises scripts, text, and HTML tags. PHP also supports several databases, including Oracle, Solid, MySQL, etc.
CFML
ColdFusion Markup Language (CFML) lets developers build dynamic web pages based on HTML. When a browser requests data in CFML, its application server automatically pre-processes the request. Therefore, it’s a fast and feasible way to develop dynamic pages that can link with any database.
FastCGI
FastCGI is an extended version of CGI; however, it’s simple, fast, and more secure than CGI. The main purpose of FastCGI is to reduce the overhead of using CGI by allowing a separate server to handle more HTTP requests.
*************************************************
After a few preliminaries, the process is to compile the "thttpd" source code which is four easy steps: 1. "make clean", 2. ".configure", 3. "make". and 4. "sudo make install" which in total took 5 minutes and 31 seconds! Moving the files to where they are supposed to be and adding the other necessary items that thttpd needs took another 45 minutes and 15 seconds to get to the first test to see if everything up to this point is working right. Almost all of this is copy and paste into the terminal and then enter it. One will have to add a user (category or header item) that needs a password, other than that it is just copy the command and paste it. The first test is to see if your web site is at "localhost".
I spent an hour at this point trying to get Chromium to come up and finally resorted to my Android phone and checked that the site was working, clicking on the first user and seeing if it opened and then clicking on "../" to see if it went back up one level. The solution: install "ELinks" then it is just "elinks http://localhost/" # when this is entered in a terminal it should bring up the web site at localhost within a couple of seconds.
Another three and a half minutes to kill thttpd, then look at the logs and add some other files on the web site. Eight and a half minutes brings us to the next test, will it auto start?
frank@AC0XL-Pi-B-Plus-1-2:/etc $ sudo ./rc.local ### sudo (Starting thttpd from rc.local, REMEMBER the sudo!!!!!!)
My IP address is 192.168.1.90
Starting thttpd.
thttpd
frank@AC0XL-Pi-B-Plus-1-2:/etc $
frank@AC0XL-Pi-B-Plus-1-2:/etc $
Checking out the new things on the web site this time using the IP Address, then checking if log-rotate works. Check out the log files, check if the "man" pages work, and add some new entries to the log by going to the web site and clicking on items from different computers or even phones. By this time it was time for me to go to bed. The computer can be shut down and when it is turned back on thttpd should be running by the time the desktop appears, as seen by its PID Numbers in "ps -e", (thttpd_wrapper, thttpd). "thttpd_wrapper" will restart "thttpd" in 10 seconds if it ever quits!
pi@AC0XL-Pi-400:~ $ # The beauty of loading "thttpd" to "/home/local/sbin/" is that one can just copy the complete "/home/" directory and all of your web site will be backed up. There is no need to worry about all of the other system files that are under "/usr/" that the computer will need to change if a different OS is installed!
pi@AC0XL-Pi-400:~ $
pi@AC0XL-Pi-400:~ $ # I will load some of my web site so one can get a feel of what will be required when moving your web site to new hardware.
pi@AC0XL-Pi-400:~ $
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 115G 94G 17G 86% /
devtmpfs 1.7G 0 1.7G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 759M 996K 758M 1% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
/dev/mmcblk0p1 253M 31M 222M 12% /boot
tmpfs 380M 32K 380M 1% /run/user/1000
/dev/sda1 253M 112M 141M 45% /media/pi/system-boot
/dev/sda2 917G 560G 321G 64% /media/pi/writable
### These are the steps I used to build my back-up of my Pi-400 since that was the only other 128GB microSD card I had and just copied the /home/ directory from my Pi-4B web server to a 1TB hard drive. I also needed some extra space so I copied the 32G0SD directory to a directory called "pulled" and copied my e-mail files ".thunderbird" there too. To rebuild Thunderbird, one just needs to install Thunderbird and skip the setup page, Then copy the old dot file (cp -av .thunderbird to ~/ ) which will put your old thunderbird files back in your home directory. ( cp -iav .thunderbird to your back-up media. Install Thunderbird, and open it, but do not run the setup, but delete it. Then replace the .thunderbird file in your home directory with your back-up copy with cp -av .thunderbird ~/ (leave the "i" off so you do not have to answer all of the over-write questions)).
# ### I just listed the commands that I used.
# df -h ### Check for space used and free space in drives.
# cd /media/pi/writable/ ### Change to new drive that will be the backup drive.
# ls -al ### Check that you are where you need to be, space for backup.
# sudo mkdir AC0XL-Pi-400 ### sudo - make directory for the backup.
# cd *400 ### Change to the backup directory.
# sudo mkdir 2022-04-08 ### sudo - date the directory for the backup.
# cd 2022-04-08/ ### Change to the backup date directory.
# sudo cp -iav /home/ . ### sudo - sudo copy all the files in /home/ to here, they do have different owners so you need sudo.
# du -hd2 ### Do a check of the backup to make sure everything is there that you want. If you use du -bd2 the count will be in bytes.
# df -h ### Do another check on how much free space is left.
#
#
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $ sudo cp -iav /home . ### Making a backup for the web server.
### /home/pi (printout text skipped)
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $ du -hd2 ### A listing of my back-up for my new copy for the Pi 400. and the Pi 4B web server.
8.1G ./pulled/.thunderbird
35G ./pulled/32G-SD
43G ./pulled
24K ./home/music
24K ./home/documents
24K ./home/pictures
24K ./home/delinquent-accounts
49G ./home/pi
1.4M ./home/local
24K ./home/va
24K ./home/searles-sav-on-propane
24K ./home/notices
24K ./home/memes
24K ./home/downloads
24K ./home/rome
24K ./home/ac0xl
24K ./home/freedom
24K ./home/videos
49G ./home
91G .
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/root 115G 94G 17G 86% /
devtmpfs 1.7G 0 1.7G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 759M 996K 758M 1% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
/dev/mmcblk0p1 253M 31M 222M 12% /boot
tmpfs 380M 32K 380M 1% /run/user/1000
/dev/sda1 253M 112M 141M 45% /media/pi/system-boot
/dev/sda2 917G 560G 321G 64% /media/pi/writable
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $
### This is with the new microSD card set up and copied the "pulled" files back to the new card.
### cd /media/pi/writable/AC0XL-Pi-400/2022-04-08
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08 $ cd pulled
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08/pulled $ sudo cp -iav 32G-SD ~/
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-Pi-400/2022-04-08/pulled $ sudo cp -iav .thunderbird ~/
pi@AC0XL-Pi-400:~ $ du -hd2 /home
24K /home/music
24K /home/pictures
152K /home/local/www
1.2M /home/local/sbin
60K /home/local/man
1.4M /home/local
360K /home/pi/.local
38G /home/pi/32G-SD
4.0K /home/pi/Pictures
4.0K /home/pi/Music
40M /home/pi/.mozilla
368K /home/pi/.config
4.0K /home/pi/Templates
64K /home/pi/Desktop
112K /home/pi/.pp_backup
112M /home/pi/RP2040
4.0K /home/pi/Documents
8.3G /home/pi/.thunderbird
1.3G /home/pi/.cache
4.0K /home/pi/Videos
35M /home/pi/Bookshelf
4.0K /home/pi/Public
3.0G /home/pi/Downloads
20M /home/pi/freedom
51G /home/pi
24K /home/ac0xl
24K /home/va
24K /home/notices
24K /home/downloads
24K /home/documents
24K /home/videos
24K /home/searles-sav-on-propane
24K /home/rome
24K /home/delinquent-accounts
24K /home/memes
24K /home/freedom
51G /home
pi@AC0XL-Pi-400:~ $
# The safest way is to go to your destination and then copy from your source so you do not wipe out your source! "cp -iav (source) . (here)". In this case I will eliminate the "v" so I do not get a multitude of lines that will over flow my printout.
# This is copying from the backup file to a new drive (microSD card).
pi@AC0XL-Pi-400:~ $
pi@AC0XL-Pi-400:~ $ cd /media/pi/writable/AC0XL-www-Pi-4B/2022-04-08/home/
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-www-Pi-4B/2022-04-08/home $
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-www-Pi-4B/2022-04-08/home $
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-www-Pi-4B/2022-04-08/home $ ls -al
total 84
drwxr-xr-x 17 root root 4096 Feb 5 19:19 .
drwxr-xr-x 4 root root 4096 Apr 8 18:21 ..
drwxr-xr-x 16 ac0xl www 4096 Feb 10 16:54 ac0xl
drwxr-xr-x 4 delinquent-accounts www 4096 Feb 6 11:39 delinquent-accounts
drwxr-xr-x 3 documents www 4096 Feb 6 11:41 documents
drwxr-xr-x 13 downloads www 4096 Feb 6 11:45 downloads
drwxr-xr-x 10 freedom www 4096 Feb 6 11:55 freedom
drwxr-xr-x 5 root root 4096 Feb 5 17:11 local
drwxr-xr-x 3 memes www 4096 Feb 6 11:59 memes
drwxr-xr-x 4 music www 4096 Feb 6 12:01 music
drwxr-xr-x 4 notices www 4096 Feb 6 12:04 notices
drwxr-xr-x 21 pi pi 4096 Apr 8 00:40 pi
drwxr-xr-x 4 pictures www 4096 Feb 6 12:07 pictures
drwxr-xr-x 50 rome www 20480 Feb 6 12:16 rome
drwxr-xr-x 4 searles-sav-on-propane www 4096 Feb 6 12:18 searles-sav-on-propane
drwxr-xr-x 4 va www 4096 Feb 6 12:20 va
drwxr-xr-x 9 videos www 4096 Feb 11 17:02 videos
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-www-Pi-4B/2022-04-08/home $
pi@AC0XL-Pi-400:/media/pi/writable/AC0XL-www-Pi-4B/2022-04-08/home $
The rest is in "2022-04-18-AC0XL-Pi-400-thttpd-install.txt" around line 5625.
Around line 8585 the discussion starts into setting up a router so your site will be visible on the "World Wide Web".
*************************************************
The benchmark test system for thttpd, last updated 12jul98, was a 297MHz Ultra Sparc with 256MB RAM / 512MB swap running Solaris 2.6, otherwise totally quiescent. RLIMIT_NOFILE is 256 soft / 1024 hard, and v.v_maxup is 3941.
UltraSPARC-Architecture.txt
Last Updated : 22 Mar, 2022
UltraSPARC Architecture belongs to the SPARC (Scalable Processor Architecture) family of processors. This architecture is suitable for wide range of microcomputers and supercomputers. UltraSPARC is example of RISC (Reduced Instruction Set Computer). UltraSPARC architecture:
Memory: Memory consists of 8 bit-bytes. Two consecutive bytes form a halfword, four bytes form a word, eight bytes form a doubleword. UltraSPARC programs operates on Virtual Address Space (264 bytes). Virtual Address Space is divided into pages and these pages are stored in the physical memory or on disk.
Registers: UltraSPARC architecture include a large file of registers that have more than 100 general purpose registers. Any procedure can access only 32 registers only. The SPARC hardware uses window into registers file to manage all the operations of different procedures. Beside these register files, UltraSPARC also uses Program Counter, code register, and other control registers.
Data Formats:
Integers are stored as 8-, 16-, 32-, or 64-bit Binary numbers.
Characters are represented using 8-bit ASCII codes.
Floating points are represented using three different formats namely single-precision format, double-precision format, quad-precision format.
Instruction Formats: SPARC architecture use three basic instruction formats. All the instructions are of 32-bit long and first two bits are used to identify which format is being used.
Format 1- Used for Call instruction.
Format 2- Used for branch instructions.
Format 3- Used by all the remaining instructions like register load and store.
Where,
n=Indirect mode,
i=Immediate addressing,
x=Index addressing,
b=Base addressing,
p= Program counter,
e=Exponential addressing
Addressing Modes: Operands in memory are addressed using one of the following three modes:
Mode Target address(TA) calculation
PC-relative TA=(PC) + displacement
Register indirect TA=(register) + displacement
with displacement
Register indirect TA=(register-1) + (register-2)
indexed
PC-relative is used only for branch instructions.
Instruction Set: This architecture have less number of instructions as compared to CISC machines. The only instructions that access memory are load and stores. All other instructions operates on register only. Instruction execution on a SPARC system is pipelined which means while one instruction is executed next one is being fetched from memory and decoded.
Input and Output: Communication between I/O devices and SPARC operation are accomplished through memory. Input and Output can be performed with the regular instruction set of the computer, and no special I/O instructions are needed.
*************************************************
It appears the UltraSPARC comes set up for 32 bit operation:
Next, it is needed to tell the system to ignore old CPU warnings. Log in as root. Start a text editor, from GUI or use vi (if you know key functions :) ), and open file:
/platform/sun4u/boot.conf
Sometimes this "sun4u" may be different according to your platform (e.g. sun4m for SPARCStation 20). Now find and uncomment the line:
ALLOW_64BIT_KERNEL_ON_UltraSPARC_1_CPU=true
If it is not present, type it manually but in Solaris 7 it should be present, but commented (# as first character). Now save the file, reboot and you have a 64-bit system.
It appears the UltraSparc IIi 300 MHz processing four operations per clock cycle should have a BogoMIPS of around 1200. "thttpd" was designed and optimized for computers with limited resources.
On the second run of "wget" I added "--http-keep-alive" which is the default, thinking that might remove the "Connection: close" in the wget "logfile.txt" file, but it is probably just an indicator that the file was downloaded to completion, and caused "wget" to take longer. The second run took a little longer, (02:28:31 or 8911 seconds vs. 8638 seconds) to download the 42 GB (AC0XL-Pi-B-Plus-1-2-web-size.txt -- 44,016,524,750 bytes in /home/local/www/users/) on the "thttpd" web server running on a Raspberry Pi B Plus computer running at 700 MHz with 512 MB RAM connected to my Gigabit router with an ethernet cable then with an ethernet cable to my Raspberry Pi 400 onto a USB 3.0 1 TB hard drive. The router also has my Raspberry Pi 4B with 4 GB RAM running my website, several Android computers, and a Wal-Mart ONN Roku TV which was being used for the monitor for the Pi B Plus, since it has three HDMI ports. The extra 273 seconds could have been from just other activity on the router or the added "--http-keep-alive". That still leaves a through put of at least 5 MB/sec which is probably 40 times faster than my current 1 megabit/sec upload rate! A 32-bit OS is limited to less than 2 gigabytes in "thttpd" while a 64-bit OS is basically unlimited, with no changes being made in thttpd! The second run is adding almost one gigabyte to "freedom" but for now I think it is necessary to see some of what is possible with "thttpd" since it was designed for computers with limited resources. "wget" is adding the index.html and index.html.orig files.
Since "thttpd" is bullet proof out of the box and is simple to configure, I feel it would be ideal for students who might not be interested in robotics, but more interested in a web server that reaches the world, and not just their local home environment!
If the Wi-Fi ports on the Raspberry Pi Zero W and 2W are similar in speed to the ethernet Port on the Raspberry Pi B Plus then the Pi Zero W should work great for a thttpd WWW server limited to files less than 2 GB since it is only 32-bits, and the Pi Zero 2W with four cores and 64-bits should be FANTASTIC!
Your own World Wide Website for pennies!