PM Software Stack

PMSS, short of PM Software Stack is a compilation of scripts to manage single server side seedbox configuration, turning instantly and conveniently an regular Debian 7.0 server into a seedbox. The PMSS does not contain multi-server management utilities nor clustering support out-of-box, but does include PM Master GUI automatic remote update features. Debian 7 support is considered stable, Debian 8 support is considered testing/unstable. PMSS supports rTorrent+ruTorrent stack.

General features

• Semi-automatic installation
• Fast and easy creation of new user accounts with quota and rTorrent resource limits, etc.
• Easy user resource reconfiguration
• Autonomous, idempotent Lighttpd & Nginx configuration
• Single Lighttpd process per user with custom config per user (including php config)
• Public web hosting directory availability for users (/public-USERNAME/)
• Autonomous configuration of basic server features
• Compiling rTorrent + libTorrent + Quota + XMLRPC from source.
• Multiple redundancy levels for rTorrent
• Lighttpd redundancy / autorestart
• Basic server optimizations to maximize performance
• Terminate, Suspend/Unsuspend users
• Webdav for users
• Other basic management scripts

Using PMSS

See Category:PM Software Stack Guides

Changelog

Changes 2017

07/01/2017

• cron/cgroup.php: Do not execute currently, not complete
• cron/diskIostat.php: Some commenting and slight refactor
• cron/trafficStats.php: refactor, commenting, couple typo fixes
• cron/checkInstances.php: Slight refactoring, some commeting and clarifying
• cron/checkQuotas.php: remove API calls, that method is deprecated
• cron/trafficLog.php: adding a few code comments, increase temp file entropy

Changes 2016

29/12/2016

• skel/rutorrent/plugins/autools/plugin.info: add to info note for watch dir breaking things
• plugins/autotools/lang: added warnings to en, de, fi, es
• install.sh: add zip pkg
• install.sh: add -j make parameter
• util/update-step2.php: add -j make parameter
• util/update-step2.php: add zip pkg
• addUser.php: comment out retracker stuff

07/12/2016

• util/update-step2.php: (Re)install git, some updates on pkg management on deb8
• util/update-step2.php: Fix vnstat not updating on deb8
• util/update-step2.php: Install Acdcli for amazon cloud drive access
• cron/diskIostat.php: fail if no serial block devices found
• cron/diskSmart.php: Added to collect smart data of disk drives

27/10/2016

• cron/diskIostat.php: Include server timestamp on the results
• config/root.cron: Collect IOStat results every 5mins instead of 15mins (2min average)

01/10/2016

• util/update-step2.php: Add pigz (parallel implementation of gzip, [1]) to be installed
• util/update-step2.php: Small refactoring on apt package installs

25/09/2016

• lib/update.php: ruTorrent config add setting top directory correctly, filemanager plugin for example requires this

08/09/2016

• util/update-step2.php: Add Jessie (Deb8) repors
• util/update-step2.php: Add OpenVPN config for Jessie, OpenVPN config now distro specific
• util/update-step2.php: Remove some old (deb5/6) apt-get instructions
• util/update-step2.php: Debian 8 ffmpeg install from backports added
• util/update-step2.php: znc package selection for Deb8

05/09/2016

Preliminary support for deb8, most things work.

• install.sh: Lots small changes to add support for deb8
• util/update-step2.php: Added headers to launch from cli

03/09/2016

• util/update-step2.php: Install funcsigs pip package, slight update to upgrade list of packages for flexget

03/08/2016

• Small visual updates

25/07/2016

• util/update-step2.php: Removed some apt-get instructions as they are in install.sh already
• install.sh: Some slight refactoring on the package installs
• util/update-step2.php: Install rclone
• cron/diskIostat.php: Make iostat current stats available also via direct http download

21/02/2016

• checkInstances.php: Typo fix
• checkLighttpdInstances.php: Typo fix

07/02/2016

• terminateUser: Comments, multiple killall and userdel directives as procs sometimes just don't want to die ... or are being relaunced. TODO check why this actually happens instead of this hack
• cron/checkInstances.php: If user is suspended, skip trying to start rTorrent ... Infact kill all procs :)
• cron/checkLighttpdInstances.php: Same thing here

28/01/2016

• cron/trafficLimits: Some refactoring, adjusting limits for new speed caps and added code to choose limit level based on total traffic limit as well (512K/s allowed to do more than limit on minimal plans)
• lib/rtorrentXmlrpc: Add function for setting download rate
• cron/trafficLimits: Restrict download speed as well IF entry plan (less than 500GiB traffic limit) to 20Mbps

24/01/2016

• update-step2: Install libffi-dev, python-dev, pyopenssl ndg-httpsclient pyasn1 cryptography for better flexget support
• update-step2: replace some nameservers with sed to OpenDNS (Online.net is censoring some trackers)

14/01/2016

• update-step2: Removed creating /etc/autodl.cfg, instead removing it, as it's not a prerequisite for autodl anymore and we can allow all permissions.

11/01/2016

• addUser.php: Set also user homedir .trafficLimit file (so we can display it in GUI), slight refactor on this section
• util/userTrafficLimit.php: Same thing as above
• (GUI) Stats: Added traffic limit number
• (GUI) Welcome page: Added meter for traffic limit

05/01/2016

• util/update-step2: Initially typoed filename as tesfile, remove that and create the correct file
• util/update-step2: Create /var/www/testfile, 100M random data

01/01/2016

• cron/diskIostat: Added history logging raw output (We are seeing IOPS outputs of 1000+ from magnetic drives, so need debug data for that)
• cron/diskIostat: History logging wrong output mode (should've been append).

Changes 2015

30/12/2015

• update-step2: Refactoring the openvpn setup section, which also bug fixes the situation where openvpn is setup but configs for users were not made

28/12/2015

• upate-step2: Bug fix: cgroup-bin apt-get did not have autoconfirm, requiring human intervention to complete update

26/12/2015

• cron/trafficLog: Log www-data & unmatched traffic as well
• cron/trafficStats: Parse for all log files found on logging dir, so includes unmatched, www-data, terminated users etc.
• cron/trafficStats: Log total data for the month period
• cron/trafficStats: Removing legacy code, small refactoring and commenting bits
• showTraffic: Show for all data files found
• lib/traffic.php: saveUserTraffic now saves to /var/run/pmss/trafficStats, and creates that directory if it does not exist
• lib/traffic.php: Bug fix: Skipped if data exceeded 7500Mb in 5mins... that's only 25MB/s. Now limit is 500MB/s with 150 000MiB check.
• showTraffic: Now uses /var data files. Instead of skipping on missing, throw an error
• showTraffic: If data is 0 for the month, skip displaying this user (gone already)
• util/makeMonitoringRules.php: Add 'www-data' to list of users
• showTraffic: Display month total across all users in TiB
• util/update-step2: Fix vnstat configuration typo to set the maximum link speed to 1Gbps
• cron/cgroup.php: Created, initial version. Needs quite a bit of refactoring, testing level right now.
• util/update-step2: install cgroup-bin, add fstab cgroup entry and mount it
• etc/seedbox/config/root.cron: Add cgroup cron job, execute once per hour

21/12/2015

• update-step2: run apt-get upgrade at the end

20/12/2015

• update-step2: Install btsync 2.2, as btsync2.2. System will have 1.4 as btsync and 2.2 as btsync2.2

19/12/2015

• update-step2: separate setup apt sources.list.d/sonarr.list. This also means Sonarr will now update normally.
• createNginxConfig.php: Now always run lighttpd config. This is a bug fix, where manually changed lighttpd port would eventually mismatch on nginx
• configureLighttpd.php: Increased potential port range.
• skel/.lighttpd/custom: Empty file created to avoid warnings. This is where users can input their custom lighttpd settings if they need some.
• userTransfer.php: Added .lighttpd/custom for migration
• userTransfer.php: Decreased the time for re-attempting user migration
• recreateUser.php: Also copy htpasswd file from old version

08/12/2015

• (GUI) Change welcome page quota display to use Kibibyte format to avoid confusion.

30/11/2015

A privileged user could use an internet download manager to basicly DOS the server with huge amount of threads trying to download large files (gigs in size), with sufficient thread quantity eventually linux kernel couldn't schedule any I/O time for disk nor ram for essential applications such as SSH. This could not be done by unprivileged user since requirement is very large file sizes. This would lead to a race condition as nginx wants to cache as fast as possible, hence "downloading" from user lighttpd process as fast as possible, which results in heavy RAM consumption and 'especially' heavy disk I/O activity - with sufficient threads scheduling breaks down.

• template.lighttpd: Introduce per connection rate limit (2048k/s) and global rate limit (12800k/s)
• template.nginx-user: Introduce user con limit to 16, rate limit (2048k/s) after 100MiB
• template.nginx-user: Introduce public con limit to 8, rate limit (512k/s) after 100MiB
• tmeplate.nginx: Add shared memory zone for connection limits

18/11/2015

• util/userConfig.php: Lower limit on filesLimit for very small disk quotas, set to 15000
• (GUI) welcome.php: Display of /etc/seedbox/config/vendorWelcome moved to top of the welcome page

12/11/2015

• cron: renice mono and ionice mktorrent added
• update-step2: install some perl modules

26/10/2015

• .lighttpd.conf: Add include_shell for .lighttpd/custom for user custom settings
• .lighttpd.conf: Add rutorrent no cache headers, and also add mod_setenv to be loaded
• rtorrent.rc template: Increased base buffers

24/10/2015

• rutorrent/plugins/hddquota: Bumped version to 1.15
• ruTorrent: Updated to version 3.7
• install.sh: Bumped rTorrent/libTorrent to 0.9.6/0.13.6, bumped xmlrpc-c to revision 2776
• rTorrent: Updated to 0.9.6/0.13.6

16/10/2015

• update-step2: Make openvpn config package even if this is not an install run
• update-step2: fix flexget install if clause

12/10/2015

• pulsedBox: Login for mds now fixed
• ruTorrent: Changed URI for css & js files to make sure cached copies are not attempted with new version
• update-step2: put back in to update ruTorrent index.html
• update-step2: Configure OpenVPN, Create certs, create client config package and provide files to users at /home/openvpn-**servername**.ovpn and /home/openvpn-**servername**.crt where as dots (.) have been replaced with dashes (-)
• update-step2: Make a tarball of OpenVPN config in skel, and provide it to userspace as well.
• setupNetwork.php: Create OpenVPN masquerade rules
• (GUI) welcome.php: Add beta disclaimer to ownCloud
• update-step2: Oops, openvpn config should probably reside under www
• update-step2: trim hostname before using for filename
• update-step2: flexget installation string match fix

11/10/2015

• update-step2: Install mkvtoolnix, add mkvtoolnix repo
• userConfig: Refactoring on ruTorrent stuff
• update-step2: Refactoring on ruTorrent stuff
• skel/www/rutorrent: Updated to latest with new plugins
• etc/seedbox/config/template.rutorrent.config: Updated to new
• etc/seedbox/config/template.rutorrent.access: Updated to new
• lib/update: Add rutorrent config function
• userConfig: Change to using that function
• update-step2: Reconfigure ruTorrent
• update-step2: Reordering all together everything related to ruTorrent on per user basis and some refactoring
• ruTorrent Update Latest version, should migrate settings etc. and reconfig properly :)
• update-step2: removal of removing check port plugin. Seems to be working ok again?

10/10/2015

• update-step2: Install flexget, buildtools32, python-pip

09/10/2015

• install.sh: Slight reordering of package installs to make initial setup even faster (human time wise)
• install.sh: Add lftp to be installed
• update-step2: Add lftp to be installed

04/10/2015

• php.ini: Default memory max increased to 512MiB, upload max filesize to 16GiB and max simultaneous uploads to 50

02/10/2015

• addUser.php: refactoring and adding a chown to make sure everything is setup allright in regards of retrackers

01/10/2015

• update-step2: Another fix for retrackers related stuff
• userPermissions.php: Couple small fixes related to the above

30/09/2015

• Small update to rTorrent + libtorrent, faster tracker check interval default setting, and close idle connections faster
• install.sh: Refactoring to have all questions at the beginning
• install.sh: Added many of the pkgs from update-step2 here, so we can later on remove them from update-step2
• util/setupNetwork.php: commented out link speed determination - not required
• install.sh: sysfs added net.ipv4.ip_forward = 1
• setupNetwork.php: added enabling ipv4 ip forward
• userPermissions.php: path fixes
• addUser.php: Typo fix
• (GUI) welcome.php: Updated with owncloud install link, and removed the need to visit setup-owncloud-finish.php
• (GUI) Frames: Added owncloud tab if present
• skel/www/setup-owncloud-finish.php: removed
• update-step2: removed setup-owncloud-finish.php, and made owncloud setup file insertion conditional
• (GUI) welcome.php: Organized the download links and fixed them :)
• (GUI) welcome.php: Fixed quota warning var giving notices on error log
• (GUI) stats.php: Changed getting IP to remote service

29/09/2015

• update-step2, addUser.php, userPermissions.php: Continuum of the earlier, apparently there's a slight logic error on ruTorrent preventing torrents directory from being created

28/09/2015

• update-step2: Fixed permissions bug introduced by yesterday's changes
• addUser.php: Same fix as above
• userPermissions.php: Fixing retroactively these. This is executed on server update.

27/09/2015

• checkInstances: Killing possible rTorrent processes fix.
• root cron: Change check Lighttpd for every 5 mins, delay beginning 30 secs
• error-502: Wording change
• addUser & update-step2: Adding retracker plugin config

04/09/2015

• createNginxConfig: Trim port number added
• configureLighttpd: Increased potential port range
• terminateUser: Remove lighttpd port config and nginx config
• terminateUser: Some wording changes shown to user.

30/08/2015

• Nginx: Custom 502 error page
• util/createNginxConfig.php: Add template for proxy_params
• Nginx: Increase proxy read timeout to 120s
• skel/.lighttpd/php.ini: Increase max execution time to 90s and increase max memory to 256M per process! Increase Max file upload size from 2M to 32M, POST max size from 8MB to 32MB
• update-step2: Fix owncloud setup files path.
• update-step2: Add php.ini to the files to be updated
• update-step2: Install packages flac, lame, lame-doc and mp3diags

11/08/2015

• cron/diskIostat.php: Add disk quantity
• cron/diskIostat.php: fix output camelCase
• Added ownCloud installers - Install is now very simple and easy!
• template.lighttpd: Added ownCloud directives for security
• util/update-step2.php: Added update-rc.d lighttpd remove
• util/update-step2.php: Added copying latest ownCloud setup files
• cron/diskIostat.php: bug fix on device selection, on some edge cases the grouping was chosen incorrectly

10/08/2015

• cron/diskIostat.php: Added cron job to gather current iostat info, 120 sec data
• cron.d/pmss: Run diskIostat every 15 mins
• util/userPermissions.php: Fixed xargs argument bug for delimiter
• cron/diskIostat.php: Added logging to -history file as well for historic data, for potential later needs

02/06/2015

• userPermissions.php: Find all directories under user account and chmod them to 751 (default directory permissions: rwxr-x--x).

06/04/2015

• userPermissions.php: Chmod .ssh if the directory exists

05/04/2015

• userPermissions.php: Chowning as well to make sure right user & group owns the directory.
• updates-step2.php: Added ZNC and ZNC-EXTRA packages

02/04/2015

• configureLighttpd.php: Typo fix

01/04/2015

Some bug fixing to yesterday's release

• userPermissions.php: chmod 771 couple temp dirs for each user's lighttpd
• update-step2.php: apt-get remove go git mercurial added since these packages are not needed

31/03/2015

This release is all about converting to per user lighttpd processes and several bug fixes, and hence enhancing service quality and security while opening the door for future new feature developments. This also adds capability for public web hosting from the seedbox.

• install.sh: remove packages golang git mercurial
• update-step2: remove packages golang git mercurial
• install.sh: also put into fstab vfsv1 instead of vfsv0
• update-step2: remove migration to vfsv1 since this is prone to breaking quota data. need to look for better migration method.
• etc/seedbox/config/template.lighttpd added and created
• etc/skel/lighttpd directory added
• etc/seedbox/config/template.nginx-user added and created
• etc/seedbox/config/template.nginx added and created
• util/configureLighttpd.php: Created configurator for per user lighttpd configs
• etc/skel/lighttpd/compress directory added
• etc/skel/lighttpd/upload directory added
• template.lighttpd: both password protected (normal GUI) and public web directories included in the config where user can run whatever they please.
• changePw.php: changes now the per user htpasswd file (username/.lighttpd/.htpasswd)
• addUser.php: use changePw.php script instead of set in the file
• addUser.php: if password rand has been passed, use a random password.
• addUser.php: remove chmod rules
• addUser.php: if runtime/trafficLimits dir does not exist, create it
• addUser.php: use configureLighttpd.php script instead of setting it up hardcoded within the script
• addUser.php: change expire date to 2100-01-01
• addUser.php: Do not add user to www-data group
• util/createNginxConfig.php: Create per user proxy config files (idempotent) and set default nginx config from template
• createNginxConfig: Create SSL config.
• template.nginx: Include SSL config
• scripts/startLighttpd added and created
• scripts/cron/checkLighttpdInstances.php added and created
• update-step2: Added all the lighttpd configuring, installing packages, managing processes etc. there. This should be the bulk of conversion
• addUser.php: create nginx config, and restart nginx
• config/root.cron: Comment out checkHttpd for now and add checkLighttpdInstances in there
• userConfig.php: Do not launch traffic limit script if none has been passed
• lib/rtorrentConfig.php: If no logger, do not echo the contents (rel: addUser.php needlessly echoing rtorrent config)
• changePw.php: if password defined it's not set to correct variable and thus cannot set password
• changePw.php: Also check for existence of per user htpasswd file!
• util/checkUserHtpasswd.php: Created to move the user's password from global to per user.
• update-step2: run checkUserHtpasswd.php and restart all lighttpd processes
• update-step2: lots of permissions updates
• update-step2: remove some deprecated, commented out code
• util/userPermissions.php: created this to set all the folder and file permissions for the user directory. Now they are in one place!
• addUser.php: use userPermissions.php
• update-step2: use userPermissions.php
• template.lighttpd: Use per user php.ini and change fcgi settings a bit
• skel/.lighttpd/php.ini added
• configureLighttpd.php: Add .lighttpd dir if it doesn't exist yet, separately check for php.ini as well.
• update-step2: Do not remove autotools plugin if it is present
• update-step2: add package ntp
• checkUserHtpasswd.php: Chown the newly created .htpasswd file
• userPermissions.php: set htpasswd file perms as well
• update-step2: move earlier in the process to make the move to nginx etc. for working conversions. make sure lighttpd processes are stopped.
• update.php: remove recreateLighttpdConfig & restart, replace with nginx
• release packager: include .lighttpd directory into the skel package.
• userPermissions.php: some chowns and extra rules required for smooth conversion
• terminateUser.php: replace lighttpd related commands with nginx replacements
• terminateUser.php: remove deleting htpasswd. Add deleting home dir in case userdel does not remove it
• recreateUser.php: Changed to fit the new model.
• configureLighttpd.php: Also check for public html directory
• update-step2: Do the configs twice to be certain, as another update step might have affected.

07/03/2015

• quotaFix.php: added -n flag for quotacheck to automatically answer questions
• install.sh: Update quota vfsv0 to vfsv1 to support large quotas
• update-step2: Update quota vfsv0 to vfsv1 to support large quotas, and commit change + execute quotaFix
• update-step2: Check if user is suspended, if so do not make updates
• update-step2: Reinstalling unpack plugin for ruTorrent which was earlier removed for abuse causing extremely high loads on a server

27/02/2015

• update-step2: moved the location of nzbdrone repos to the normal repo list, cannot move sonarr because https transport needs to be first installed.
• update-step2: Refactored stopping & disabling services not needed for "global" user
• update-step2: added apt-get -f install -y to fix potentially broken or missing dependencies
• update-step2: Forgot to add -y toggle for sabnzbdplus package install requiring manual intervention

26/02/2015

• install.sh: Added locate apt package, since some minimal installs are missing this
• update-step2: Added locate apt package
• install.sh: Added python-pycurl, python-crypto, python-central packages
• update-step2: Install pyLoad and dependencies python-pycurl, python-crypto, python-central, golang, git, mercurial
• install.sh & update-step2: Install packages gcc g++ gettext python-cheetah curl fuse glib-networking libglib2.0-dev libfuse-dev apt-transport-https
• update-step2: Compile & install Megatools from source
• update-step2: Re-ordered package installs as some installation & compilations were attempted BEFORE installing the dependiences.
• update-step2: get, compile and install Mono 3.8.0
• update-step2: Install Sonarr / Nzbdrone. To start it execute mono /opt/NzbDrone/NzbDrone.exe and then browse to http://servername.pulsedmedia.com:8989 - remember to change port and set security + authentication credentials!
• update-step2: To make sure they are not running for non-specific seedbox user, stop btsync and pyload, and update-rc.d disable them
• update-step2: Install sabnzbdplus

02/02/2015

• update-step2: ntpdate for some reason was not installed by default - included this on update
• update-step2: now vnstat config is checked on every update
• update-step2: vnstatConfig also sets maxbandwidth higher
• apt-get clean & update for some reason missing from update.php - added, and also temporarily for step2

31/01/2015

• Security update on lighttpd config.

30/01/2015

• btsync sample config now in skel as .btsync.conf-sample
• btsync binary added to /usr/bin/btsync now.

21/01/2015

• rutorrent/share/settings/rss folder was missing, created to skel and added to update for creating it

19/01/2015

• trafficStats: Added logging also date & time. For some reason this was not logged.
• trafficLog: removed serverApi calls as cnc is not utilized anymore
• skel/www/rutorrent/plugins/check_port removed as well. was prior removed only on update. this plugin doesn't function anymore as expected.
• some debug logging added for trafficStats cron

Changes 2011-2014

Can be found at PM Software Stack Changelog 2011-2014