OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: postfix shared lib, dynamic tables and others
From: Michael Ju. Tokarev (mjttls.msk.ru)
Date: Mon Oct 02 2000 - 01:50:58 CDT

Hello!

Some time ago this topic was discussed on the list.
And here this goes.
No, I have not finished all the things, but instead
have a bunch of questions... :)

The "idea" was:
 a) to allow postfix to be built using common shared
    library
 b) to use dynamic maps in postfix.

Both only on systems where this is possible (one to
mention where this is _not_ possible is Interactive
UNIX from Sun, dead now). And both with a compile-time
choice.

The first problem that I encountered was the global
vars. Here goes two patches that just moves some
variables to different files:
  debug_peer_level, debug_peer_list moved to
    global/debug_peer.c
  command_maxtime moved to mail_params.c
(ok, one change at a time!)
There are two patches actually -- the second one
moves initialization of command_maxtime to the
global table in mail_params.c -- I'm not shure if
this is "right" to do but think so.
Both patches does not introduced new functionality.

And follows a list of little proposals.
And I have postfix snapshot-20000924 built and
running using shared library "libpostfix.so.20000924"
(built by hand, using dirty hacks) -- it is _mach_
smaller than original one, and it is actually
smaller than sources (was bigger) -- it is on linux
i386 (first column - size of file with .so, second -
standard size):

  175084 - libpostfix.so.20000924

   14240 78492 bounce
   19704 97436 cleanup
    4280 70908 error
   19892 91772 lmtp
   30940 115516 local
   21552 58172 master
   29620 95932 nqmgr
    7412 71260 pickup
    9760 85468 pipe
   25396 91676 qmgr
    7200 67900 showq
   23240 94908 smtp
   38140 119420 smtpd
    5364 63868 spawn
    9232 70748 trivial-rewrite

    8140 51036 postalias
    5960 19964 postcat
   20756 55516 postconf
    6868 49788 postdrop
    5420 33692 postfix
    4848 35420 postkick
    5600 35036 postlock
    5420 34972 postlog
    7204 46140 postmap
    7920 40156 postsuper
   13104 59164 sendmail
    7364 26108 smtp-sink
   12232 31068 smtp-source

1. I'd like to split master directory onto two --
one for libmaster and one for master daemon, --
to simplify building of shared library. As we
concluded, there should be one single shared
library for all the postfix programs, so that
library should be built before any other program,
but in master dir there are a library (that should
be part of "libpostfix.so") and a program that
uses that library).

2. With this split, we'll have two dirs -- master
and libmaster. For convience, I'd suggest to
rename other library directoried to have 'lib'
prefix: libglobal, libutil, libdns (and libmaster).
Having this common naming convension, there will
be simpler to go further. "Problem" here is that
libpostfix.so should be built from several subdirs,
and it will be better if them will have common names.
This both rather trivial, but I want an "approve"
from Wietse here, as this is a source organization
things, where Wietse is a god.

3. I think that .so library should be named using
version of postfix (while it is in beta/snapshot
stage, it should be something like
libpostfix.so.snapshot-yyyymmdd or
libpostfix.so.yyyymmdd.p, where p is a patchlevel,
and something like libpostfix.so.major.minor for
release), to allow testing etc and enshure that
correct library used. Is this ok?

4. The most tricky thing is _where_ to place shared
lib and how to find it. I'd suggest to use standard
system-wide libdir (like /usr/lib) for this (since
at least by default postfix uses /usr/libexec,
/usr/bin, /usr/sbin), and not to hardcode any
additional LD_RUN_PATH to executables. But this
will break non-default installations like placing
all stuff into /opt/postfix-NNNN. The simplest
and good way here is to know destination at compile
time. See also 6. Anyway, for things to work
before installed (postconf used here) all executables
should be a shell-wrappers like those that libtool
have (postconf is a two-line shell script and
postconf.bin is a real exe).

5. For some tools it may be ok to _not_ use
shared lib -- especially for debug/bench tools
like smtpstone and fsstone. Is this ok?
I think that for this, making both .so and static
version is a good idea.

6. Wietse, why not to switch to something like
combined automake+autoconf style for supporting/
generating makefiles? I already asked about this,
but got no answer (no, I'm not about using that
tools). My suggestion:

 in each subdir have a script like this
 (this is a shell script with just vars):

   NAME=master
   SOURCES="master.c master_conf.c master_ent.c
      master_sig.c master_avail.c ..."
   DEPS="libmaster libglobal libutil"
   HEADERS="mail_server.h master_proto.h"
   OTHER="master.h" # for `dist' target
   TEST_SOURCES="" # for test targets

  called "descr.sh", and optional Makefile.in.bot
  file that, if exists, will be concatenated
  to resulting makefile.

With this, and having trivial makemakes shell
script, it is very simple to support the whole
thing, and to allow other goodies (like building
in other object directories etc).

Also, what's about having real `configure' script?
It should process options, check system type and
generate makefiles (as autoconf does). This will
simplify life for people building postfix, and
simplify maintenance also. Consider:
  ./configure --with-maps="pcre db mysql" \
    --with-mysql-includes=-I/usr/local/mysql \
    --libexecdir=/usr/lib/postfix \
    --with-sendmail=/usr/other/sendmail \
    --with-confdir=/etc/mail \
    --setgid=postdrop \
    --man8ext=1m \
    --use-sharedlib \
    ...
After this, simple `make install' will install
things (or `make install DESTDIR=/some/other/place').

Scripts are trivial, and I can wrote 'em easily.
But again, I need an "approve" (or explanation why
this is wrong/bad).

Without this (esp. about auto-generating makefiles)
it is hard to support both libpostfix.so and
dynmaps.

7. Does anybody knows if there is a system out
here that have .so support (and dlopen-like) that
needs .so dependances to be linked with executable?
I.e. if one leaves -lresolv -ldb etc to link of
libpostfix.so and omits them from each program,
is there a system where this will not work?
The same is for dlopen() -- if we have say
dbmap.so that is linked with -ldb (and no other
references to -ldb exists) -- is there any system
on that loading of this dbmap.so will not work
without explicit loading of -ldb first? 

--
That's all for a first time.

Patches follows (I made 'em staying in src/
subdir of postfix, so that 'em can be applied
(partially) to official release).

Regards,
 Michael.

--- global/debug_peer.c.orig Fri Dec 11 21:55:27 1998 +++ global/debug_peer.c Mon Oct 2 04:33:16 2000 -70,6 +70,9 #include <namadr_list.h> #include <debug_peer.h> +char *var_debug_peer_list; +int var_debug_peer_level; + /* Application-specific. */ #define UNUSED_SAVED_LEVEL (-1) --- global/mail_params.c.orig Fri Sep 22 23:58:21 2000 +++ global/mail_params.c Mon Oct 2 04:36:57 2000 -58,6 +58,7 /* int var_ownreq_special; /* int var_daemon_timeout; /* char *var_syslog_facility; +/* int var_command_maxtime; /* /* void mail_params_init() /* DESCRIPTION -164,6 +165,7 int var_ownreq_special; int var_daemon_timeout; char *var_syslog_facility; +int var_command_maxtime; /* check_myhostname - lookup hostname and validate */ --- lmtp/lmtp.c.orig Sun May 7 23:57:29 2000 +++ lmtp/lmtp.c Mon Oct 2 04:31:46 2000 -228,8 +228,6 int var_lmtp_data1_tmout; int var_lmtp_data2_tmout; int var_lmtp_quit_tmout; -char *var_debug_peer_list; -int var_debug_peer_level; int var_lmtp_cache_conn; int var_lmtp_skip_quit_resp; char *var_notify_classes; --- local/local.c.orig Sun Jun 25 19:13:03 2000 +++ local/local.c Mon Oct 2 04:38:57 2000 -426,7 +426,6 char *var_allow_files; char *var_alias_maps; int var_dup_filter_limit; -int var_command_maxtime; char *var_home_mailbox; char *var_mailbox_command; char *var_rcpt_fdelim; --- pipe/pipe.c.orig Thu Sep 21 04:45:35 2000 +++ pipe/pipe.c Mon Oct 2 04:39:33 2000 -241,12 +241,6 #define PIPE_FLAG_SIZE (1<<4) /* - * Tunable parameters. Values are taken from the config file, after - * prepending the service name to _name, and so on. - */ -int var_command_maxtime; /* system-wide */ - - /* * For convenience. Instead of passing around lists of parameters, bundle * them up in convenient structures. */ --- smtp/smtp.c.orig Mon Sep 18 05:24:31 2000 +++ smtp/smtp.c Mon Oct 2 04:33:43 2000 -224,8 +224,6 int var_smtp_data2_tmout; int var_smtp_quit_tmout; char *var_inet_interfaces; -char *var_debug_peer_list; -int var_debug_peer_level; char *var_notify_classes; int var_smtp_skip_4xx_greeting; int var_smtp_skip_5xx_greeting; --- /usr/mjt/snapshot-20000924/src/smtpd/smtpd.c.orig Tue Aug 22 04:43:08 2000 +++ smtpd/smtpd.c Mon Oct 2 04:34:01 2000 -305,8 +305,6 int var_smtpd_hard_erlim; int var_queue_minfree; /* XXX use off_t */ char *var_smtpd_banner; -char *var_debug_peer_list; -int var_debug_peer_level; char *var_notify_classes; char *var_client_checks; char *var_helo_checks; --- spawn/spawn.c.orig Thu Sep 21 04:48:31 2000 +++ spawn/spawn.c Mon Oct 2 04:40:03 2000 -118,12 +118,6 /* Application-specific. */ /* - * Tunable parameters. Values are taken from the config file, after - * prepending the service name to _name, and so on. - */ -int var_command_maxtime; /* system-wide */ - - /* * For convenience. Instead of passing around lists of parameters, bundle * them up in convenient structures. */

--- global/mail_params.c.orig Mon Oct 2 09:27:50 2000 +++ global/mail_params.c Mon Oct 2 09:21:27 2000 -298,6 +298,7 VAR_FLOCK_DELAY, DEF_FLOCK_DELAY, &var_flock_delay, 1, 0, VAR_FLOCK_STALE, DEF_FLOCK_STALE, &var_flock_stale, 1, 0, VAR_DAEMON_TIMEOUT, DEF_DAEMON_TIMEOUT, &var_daemon_timeout, 1, 0, + VAR_COMMAND_MAXTIME, DEF_COMMAND_MAXTIME, &var_command_maxtime, 1, 0, 0, }; static CONFIG_BOOL_TABLE bool_defaults[] = { --- local/local.c.orig Mon Oct 2 09:27:50 2000 +++ local/local.c Mon Oct 2 09:22:37 2000 -599,7 +599,6 int main(int argc, char **argv) { static CONFIG_INT_TABLE int_table[] = { - VAR_COMMAND_MAXTIME, DEF_COMMAND_MAXTIME, &var_command_maxtime, 1, 0, VAR_DUP_FILTER_LIMIT, DEF_DUP_FILTER_LIMIT, &var_dup_filter_limit, 0, 0, 0, }; --- pipe/pipe.c.orig Mon Oct 2 09:27:50 2000 +++ pipe/pipe.c Mon Oct 2 09:22:59 2000 -753,13 +753,7 int main(int argc, char **argv) { - static CONFIG_INT_TABLE int_table[] = { - VAR_COMMAND_MAXTIME, DEF_COMMAND_MAXTIME, &var_command_maxtime, 1, 0, - 0, - }; - single_server_main(argc, argv, pipe_service, - MAIL_SERVER_INT_TABLE, int_table, MAIL_SERVER_POST_INIT, drop_privileges, MAIL_SERVER_PRE_ACCEPT, pre_accept, 0); --- spawn/spawn.c.orig Mon Oct 2 09:27:50 2000 +++ spawn/spawn.c Mon Oct 2 09:23:10 2000 -285,13 +285,7 int main(int argc, char **argv) { - static CONFIG_INT_TABLE int_table[] = { - VAR_COMMAND_MAXTIME, DEF_COMMAND_MAXTIME, &var_command_maxtime, 1, 0, - 0, - }; - single_server_main(argc, argv, spawn_service, - MAIL_SERVER_INT_TABLE, int_table, MAIL_SERVER_POST_INIT, drop_privileges, MAIL_SERVER_PRE_ACCEPT, pre_accept, 0);