Introduction
NcFTPd now ships with its own dynamic report generation package.
This allows an administrator to view various statistics from a web browser.
This requires that there be a HTTP (web) server installed on the machine
that NcFTPd runs on. The report package consists of the following
reports:
-
Top downloaded files
-
Download summary
-
Query for a downloaded file
-
Upload summary
-
Query for an uploaded file
-
Real users activity summary
-
Query for a real user
-
Anonymous user summary
-
Query for an anonymous user
-
Client IP summary
-
Top client IP accesses
-
General Statistics* (logins, downloads, uploads,
client connection types, client program types, etc.)
-
Simultaneous Users*
*some reports require that your system have
System V IPC (shared memory). Unfortunately that means SunOS
and BSD/OS users, for example, will not be able to run these reports.
Installation
Unfortunately the report package is a bit cumbersome to install, because
there are so many programs which must be properly configured to work together.
Indeed, you may find you spent more time installing the reports than you
did installing NcFTPd.
The report package is distributed with NcFTPd 2.1.0 and later,
and is in the rptbin subdirectory. A typical installation
of NcFTPd is in the /usr/local/etc/ncftpd directory.
Therefore you should now have a /usr/local/etc/ncftpd/rptbin directory.
Tweaking the path to the .env file
If you are not using /usr/local/etc/ncftpd as your base directory,
you will need to edit all the scripts which depend on the exact pathname
to the ncftpd_reports.env file. This file is a shell script
which sets up environment variables that are shared between each CGI script.
Each script contains the line:
. /usr/local/etc/ncftpd/rptbin/ncftpd_reports.env
You need to change the path so that it points to the ncftpd_reports.env
file. For example, if your base directory is /home/ncftpd,
change that line in all the scripts to:
. /home/ncftpd/rptbin/ncftpd_reports.env
The list of scripts you need to change is all of the .cgi and
.sh files that came in the rptbin directory.
Installing the CGI scripts
The report request form and the report generation scripts require that
they be executed by the web server. Choose a dedicated directory
that allows for CGI execution -- we'll denote this RPTCGIDIR.
For most web servers, you can create a subdirectory in the cgi-bin
directory. Otherwise, you may need to configure your web server so
that it recognizes .cgi as an extension of an executable CGI program,
in which case you can create the RPTCGIDIR directory anywhere in the web
document tree.
Let's assume that your web server is installed in /home/httpd,
and you have a /home/httpd/cgi-bin directory. You would
create a /home/httpd/cgi-bin/ncftpd as your RPTCGIDIR directory.
You must then copy the rpt*.cgi script files into this directory,
and chmod and chown them appropriately so that the web user (the
UID of the web server) can run them. (Do not install the test*.cgi scripts at this time -- you only need them if you are diagnosing problems.)
If you chose to use the .cgi extension method instead, and
your document tree was /home/httpd/htdocs, you could create a
/home/httpd/htdocs/ncftpd directory as your RPTCGIDIR and install
the .cgi scripts in to it.
Creating the temporary output directory
The .cgi programs need to be able to create other web pages and
image files. This requires a dedicated directory in the web document
tree -- we'll denote this ODIR. This directory must
be writable, readable, and executable by the UID of the web server, because
the .cgi scripts write files into it and then the user's browser
reads the output back from it.
It may be tempting to use RPTCGIDIR for this -- don't do it! The
cleanup script (clean_tmpdir.sh) would end up wiping out the .cgi
programs. Also do not choose a subdirectory anywhere within the cgi-bin
tree; although that would work, that could leave your system open to a
security breach if people can find a way to upload real programs into your
ODIR.
For example, if your document root was /home/httpd/htdocs,
you could create /home/httpd/htdocs/tmp as your ODIR. You
would then make sure the directory is owned by the web user, and
is read, write, and executable by that user, for example:
chmod 755 /home/httpd/htdocs/tmp
chown web /home/httpd/htdocs/tmp
The cleanup script, when run, will remove files older than 1 day.
If you have other programs writing into your ODIR, or other data in there
that you want preserved, create a different directory.
The corresponding directory to ODIR is RELODIR, which
is what the external browser uses in URLs. For example, if ODIR was
/home/httpd/htdocs/tmp you would set RELODIR so that it omits
the actual path of the web document root, so RELODIR would be /tmp.
The web browser would then see http://yourhost.com/tmp/ in the
URLs produced.
Installing the utility programs
The .cgi scripts use some external programs that do a lot of the
grunt processing and calculating that isn't convenient to do within the
limits of a shell script. These programs are datemunger,
sum_stat_logs, logcat, form2env, reresolv,
and clean_tmpdir.sh. The directory they are placed in is
denoted RPTBINDIR, and defaults to /usr/local/etc/ncftpd/rptbin.
So, if you are using the default base NcFTPd base directory, you
probably do not need to move them.
These programs need to be run by the .cgi scripts, so therefore
they must be accessible by the web user. Make sure that the
permissions are set so that they can be run. If your web server is
running under a chroot environment, you'll need to make sure that RPTBINDIR
is accessible from that root.
Installing Gnuplot
Some of the .cgi scripts can create graphs illustrating trends
over a given time period. To enable graphics, you need to have the
freeware program gnuplot
installed. This program takes data files and plots the graphs.
However, the .cgi scripts require that your version of gnuplot
have the GIF generation code compiled in. The default distribution
of gnuplot does not include this. Gnuplot
cannot be included with the NcFTPd distribution, but you can find
a pointer to it at the NcFTP Software download
page.
Like the utility programs in RPTBINDIR, gnuplot needs to be run by the
.cgi scripts, so therefore it must be accessible by the web
user. However, it does not need to be in RPTBINDIR.
Setting permissions for the web user
As you've seen, the web server processes must have permission to use just
about everything in the reporting package. If your web server does
not run as root (or runs the CGI programs as non-root), this web
user must have permission to:
-
Read and execute the .cgi scripts in RPTCGIDIR
-
Execute the programs in RPTBINDIR
-
Read the ncftpd_reports.env file
-
Execute gnuplot (optional)
-
Read, write, and execute the ODIR
-
Read your NcFTPd general.cf and domain.cf files
-
Read your NcFTPd xfer, session, and stat
log files
You will probably need to make arrangements to read the .cf files
and log files. If the web user belongs to a group, you can
change the group of the .cf and log files to this group, and give
the group read permission to them. Otherwise you may need to give
ownership of the .cf and log files to the web user, or
make them public readable.
NcFTPd itself has options to allow you to tune the server so
that it generates new log files that are web user friendly.
In the general.cf, you can use the log-owner, log-group,
and log-umask options to set the owner, group, and umask for newly
created log files and log directories. For example, if your web
user is named web and there is a group named webgrp,
you could do this in your general.cf:
log-user=web
log-group=webgrp
log-umask=027
It is important that if your web user is non-root that you
use those options. Otherwise your reports won't be able to read the
data from newly-generated logs.
Editing the shared ncftpd_reports.env file
The .cgi scripts need to know how you configured everything above.
This is accomplished by having each .cgi script source the ncftpd_reports.env
file. Edit the file and follow the directions, setting each variable
to what you have already configured.
There are a couple of parameters that have not been mentioned to this
point, an "AWK" parameter and a
"WHOAMI" parameter.
These simply correspond to the pathnames of the awk and
whoami programs on your system. The AWK parameter may
be preset to gawk, which you may or may not have. If you do not
have GNU awk, change this to the pathname of your awk,
which is often /usr/bin/awk.
Securing the reports from the general public
You should strongly consider placing the NcFTPd reports behind a
protected area of the web server, using a password protected .htaccess
file in the RPTCGIDIR, or some other method. Most sites will not
want to allow world access to their FTP server statistics, and just as
important, since the report generators consume a significant amount of
system resources, you don't want too many people running them at once anyway.
Consult your web server's documentation for detailed information on how
you can add authentication to areas of your web document tree.
Running Reports
If everything is installed correctly, you should be able to run the report
request form, rptform.cgi, from your web browser. For example, if
/home/httpd/cgi-bin/ncftpd is your RPTCGIDIR directory, you would
use http://yourhost.com/cgi-bin/ncftpd/rptform.cgi as the URL.
This page will give you instructions on which reports you can run.
Two common problems are with the main form (rptform.cgi)
not parsing it's input correctly (i.e. a message from datemunger complaing that it could not convert the form data back into a timestamp), and
another problem with gnuplot not drawing graphs.
The first problem is resolved by upgrading to a more recent version of
the report package (specifically, an updated form2env program which
parses the CGI form data). The second problem is often resolved by
compiling gnuplot yourself.
If you still have problems, install the diagnostic CGI programs
testform.cgi (which tests the form parsing) and/or
testplot.cgi (which tests the graph plotting) into your
RPTCGIDIR.
Another issue you may run into is your browser timing out before the report
completes. Depending on the amount of log data to process, the report
may not be able to finish on time. When that happens you need to
decrease the amount of time the report covers.
Some of the reports use the utility program reresolv to convert
IP addresses into hostnames. If DNS is not configured correctly,
this may cause the report to timeout. If you have this problem, you
can get around it by replacing the reresolv filter program with
a copy of cat. Better of course would be to have DNS running,
or at least configured so that it errs out immediately so reresolv
does not hold up the report.