Difference between revisions of "PM Software Stack"

From Pulsed Media Wiki
(→‎Changelog)
 
(590 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
PMSS, short of PM Software Stack is a compilation of scripts to manage single server side [[seedbox]] configuration, turning instantly and conveniently an regular [[Debian]] 5.0 [[server]] into a [[seedbox]]. The PMSS does not contain multiserver management utilities nor [[Cluster|clustering]] support out-of-box, but does include [[PM Master GUI]] remote update features. [[Debian]] 6.0 squeeze support is still in unstable status but several production servers already do exist. PMSS currently only supports [[RTorrent|rTorrent]]+[[ruTorrent]] stack.
+
PMSS, short of [[Pulsed Media]] Software Stack is a compilation of scripts to manage single server side [[seedbox]] configuration, turning instantly and conveniently an regular [[Debian]] [[server]] into a [[seedbox]]. It is free to use for anyone, see [[Installing PM Software Stack]].
 +
 
 +
PMSS supports [[RTorrent|rTorrent]]+[[ruTorrent]], Deluge, qBittorrent, rclone etc.
  
 
== General features ==
 
== General features ==
−
* Semi-automatic [[Installing PM Software Stack|installation]]
+
 
 +
Primary design goals are reliability and ease of server management.
 +
 
 +
* Automatic [[Installing PM Software Stack|installation]]
 
* [[Adding a new user|Fast and easy creation of new user accounts]] with [[quota]] and [[rTorrent]] resource limits, etc.
 
* [[Adding a new user|Fast and easy creation of new user accounts]] with [[quota]] and [[rTorrent]] resource limits, etc.
 
* [[Changing user settings|Easy user resource reconfiguration]]
 
* [[Changing user settings|Easy user resource reconfiguration]]
−
* Autonomous, [[idempotent]] [[Lighttpd]] configuration
+
* Autonomous, [[idempotent]] [[Lighttpd]], [[Nginx]], [[rTorrent]] configuration
−
* Autonomous configuration of basic [[server]] features
+
* Single Lighttpd process per user with custom config per user (including php config)
−
* [[Compiling]] [[RTorrent|rTorrent]] + [[libTorrent (Rakshasa)|libTorrent]] + [[Quota]] + [[XMLRPC]] from [[source]].
+
* Public web hosting directory availability for users (/public-USERNAME/)
 +
* Autonomous configuration of [[server]] features
 +
* [[Compiling|Compiles]] [[RTorrent|rTorrent]] + [[libTorrent (Rakshasa)|libTorrent]] + [[Quota]] + [[XMLRPC]] from [[source]].
 
* Multiple [[redundancy]] levels for [[rTorrent]]
 
* Multiple [[redundancy]] levels for [[rTorrent]]
 
* [[Lighttpd]] [[redundancy]] / autorestart
 
* [[Lighttpd]] [[redundancy]] / autorestart
−
* Basic server optimizations to maximize performance
+
* Server optimizations to maximize performance
 
* [[Removing user|Terminate]], [[PMSS Suspending/Unsuspending user|Suspend/Unsuspend]] users
 
* [[Removing user|Terminate]], [[PMSS Suspending/Unsuspending user|Suspend/Unsuspend]] users
−
* Webdav for users
+
* Server administration tools
−
* Other basic management scripts
 
−
 
 
  
 
== Using PMSS ==
 
== Using PMSS ==
 
See [[:Category:PM Software Stack Guides]]
 
See [[:Category:PM Software Stack Guides]]
 +
 +
== Debian 8 notes ==
 +
 +
Debian 8 support is considered now EOL -- No new patches will be made for Debian 8.
 +
 +
== Debian 10 notes ==
 +
 +
Debian 10 considered stable. In production.
  
 
== Changelog ==
 
== Changelog ==
−
=== Changes 2012 ===
 
−
==== 30/10/2012 ====
 
−
'''Waiting for release'''
 
−
* Fixed var/www/index.php redirection
 
−
* Added script to create traffic monitoring iptables rules
 
−
* Created /scripts/cron
 
−
* moved updateQuotas, checkInstances, checkHttpd to cron dir
 
−
* created /var/log/pmss
 
−
* Changed checkInstances + checkHttpd logging to /var/log/pmss dir
 
−
* Refactored cron tasks to utilize /etc/cron.d '''NOTE: If you have customized root crontab this may overwrite the changes!'''
 
−
* Removed "tester" from user filtration list and added srvapi, pmcseed, pmcdn to the filtration list
 
−
* Beginnings of network management cron
 
−
* If an OVH kernel is detected on Deb Squeeze, install standard and set it as default. '''Note: No Deb5 support yet - only changed IF vanilla kernel is not installed'''
 
−
* Added packages ethtool, elinks, links, lynx
 
−
* Added Basic QoS to ensure quality of service and fairness in BW sharing - not enabled yet by default needs testing
 
  
 +
=== Changes 2023 ===
 +
 +
==== NOW ON GITHUB ====
 +
See [https://github.com/MagnaCapax/PMSS https://github.com/MagnaCapax/PMSS] work and changelog continues on there.
 +
 +
==== 01/2023 ====
 +
* /etc/skel/.bashrc: Add ~/bin to path
 +
* /scripts/util/userConfig.php: Systemd enable linger, install docker rootless + docker-compose for the end user
 +
* /scripts/lib/apps/docker.php: Added, install docker rootless related stuff
 +
 +
=== Changes 2022 ===
 +
 +
 +
==== 15/12/2022 ====
 +
 +
* /scripts/lib/apps/packages.php: add package uidmap
 +
 +
==== 15/12/2022 ====
 +
 +
* /scripts/lib/apps/packages.php: Changed curl to backports, rtorrent recently started crashing again frequently for a lot of people, so update to curl fixed this.
 +
 +
==== 15/12/2022 ====
 +
 +
* scripts/util/userTransfer.php: Exclusion list extended a little bit more
 +
 +
==== 7/12/2022 ====
 +
 +
* scripts/util/userTransfer.php: Exclusion list extended to various runtime and configs which were mistakenly being migrated over (the main issue with "catchall" type of migration)
 +
 +
==== 2/12/2022 ====
 +
 +
* scripts/util/userTransfer.php: Second step migration added ~/.lighttpd/custom.d and ~/www/public directories
 +
 +
==== 23/11/2022 ====
 +
 +
* scripts/util/update-step2.php: Remove _user_/www/phpXplorer symlink, very very old compatibility thing
 +
* skel/www/phpXplorer removed
 +
 +
==== 15/11/2022 ====
 +
 +
* scripts/util/userTransfer.php: Moved .local to last step, some refactoring+commenting, and multiple attempts on 2nd step too. Increased timers in case source server issues
 +
 +
==== 15/11/2022 ====
 +
 +
* scripts/util/userTransfer.php: No need to insert username as double anymore, conditionally assume it is the same
 +
* scripts/util/userTransfer.php: Completely new transfer, attempt to transfer everything. First beta let's hope stuff doesn't break too much
 +
* scripts/util/userTransfer.php: Increase delay between re-attempts, and increase count a bit too
 +
* etc/seedbox/config/template.user-slices-pmss.conf: Tasks max lowered to 250 processes to limit fork bombing done by some users
 +
 +
==== 19/10/2022 ====
 +
 +
* scripts/util/update-step2.php: Set systemctl user slice basic defaults file
 +
* scripts/util/userConfig.php: Create user slice configs, CPU, IO Weights and memory soft & hard limits. New additional config options for cpu + ioweight, memory soft limit is advertised memory limit and memory hard limit is twice that.
 +
* etc/seedbox/config/template.user-slice.conf: Added template for the user slice
 +
* etc/seedbox/config/template.user-slices-pmss.conf: Default user slice config to be added to systems
 +
* etc/skel/www: php files updated to latest versions  (the new theming is earlier user contribution)
 +
* etc/skel/rutorrent/plugins/theme/themes: Added themes from [https://github.com/artyuum/3rd-party-ruTorrent-Themes https://github.com/artyuum/3rd-party-ruTorrent-Themes] and made MaterialDesign the default (user contrib)
 +
* scripts/util/update-step2.php: Update themes default, and add the new ones if missing
 +
* scripts/lib/packages.php: Update to [https://mediaarea.net/fr/Repos MediaInfo repos] and install newest available mediainfo package (user contrib)
 +
* scripts/lib/packages.php: Buster backport kernel update to 5.10 series
 +
 +
==== 18/06/2022 ====
 +
 +
* scripts/lib/apps/deluge.php: Add debian packages python & python3 setproctitle for deluge installation.
 +
 +
==== 04/05/2022 ====
 +
 +
* scripts/util/update-step2.php: Install latest rc.local on each update
 +
 +
==== 30/04/2022 ====
 +
 +
* scripts/terminateUser.php: Additional username checks
 +
 +
==== 13/03/2022 ====
 +
 +
* scripts/util/userPermissions.php: Changes on .config, .tmp -- as expected there's been regressions. some other small changes too
 +
* skel/.bashrc: Default to color prompt, ls colors and set locales
 +
* scripts/util/update-step2.php: Set locales. Some installers don't setup as UTF-8 by default. Update .bashrc automatically
 +
 +
==== 12/03/2022 ====
 +
 +
* scripts/util/userPermissions.php: Fixed directory 751 to 750 permissions, and fixed home directory permissions order being incorrect. '''**Untested**''' but due to public disclosure before tested fix, we have to push this out.
 +
* install.sh: Add mount proc with hidepid '''**this will most likely break some other things now -- untested, but same reason as above**'''
 +
* scripts/util/userPermissions.php: added ~/.tmp + .config to the list of chmods as well -- these are relatively new directories by new additional software added as of late, and those defaults might not be the best
 +
* addUser.php: Leave userPermissions run into the background, do not lock the process.
 +
 +
==== 06/03/2022 ====
 +
 +
* scripts/cron/checkDelugeInstances.php: Refactoring and 2 typoes on execution commands
 +
 +
==== 21/02/2022 ====
 +
 +
* GUI: Custom frames support! ~/.customFrames file with format, one per line: appname|tooltip|label|url  -- full documentation on ~/www/index.php on the format. User contribution.
 +
* etc/seedbox/config/template.qbittorrent.conf: Change for random external port
 +
* scripts/util/userConfig.php: Only config qBittorrent when config does not exist -- may cause an issue when reconfiguration is needed tho
 +
* GUI: info page stats: RAM usage reformatted for nicer look
 +
 +
==== 20/02/2022 ====
 +
 +
* GUI: Layout changes. User contribution.
 +
* GUI: Fixed height issues and removed news tab
 +
* GUI: Welcome page now has announcements rss feed
 +
 +
==== 19/02/2022 ====
 +
 +
* etc/seedbox/config/root.cron: Added bash, deluged, qbittorrent-nox, xmrig CPU & I/O scheduling rules
 +
* skel/.bashrc: added arrinfo alias for listing URLs if using the community script to install everything. User contribution.
 +
* www/lighttpdRestart.php: Change lighttpd restart to SIGUSR1 -- faster and cleaner. User contribution
 +
 +
==== 15/02/2022 ====
 +
 +
These are user submitted
 +
 +
* util/configureLighttpd.conf: Create .lighttpd/custom.d
 +
* skel/.lighttpd/custom.d: directory added
 +
* util/update-step2.php: chown some log files, and bins for admin access only
 +
* seedbox/config/template.lighttpd: Add include .lighttpd/custom.d/*.conf for user customization purposes
 +
 +
==== 11/02/2022 ====
 +
 +
* Welcome page: Changed rTorrent restart button to middle and added lighttpd restart button
 +
* seedbox/config/template.nginx-user: Add 502 error page directive
 +
* var/www/error-502.html small text update
 +
 +
=== Changes 2021 ===
 +
 +
==== 20/12/2021 ====
 +
 +
* scripts/util/userPermissions.php: Reverted chown command back
 +
 +
==== 16/12/2021 ====
 +
 +
* scripts/util/userPermissions.php: Remove -R from chmod 640 user home dir -- this was potentially messing up some emby & jellyfin installations -- also remove an chown -R directive (redundant).
 +
* config/template.rutorrent.config: Increase timeouts and change log file
 +
 +
==== 29/11/2021 ====
 +
 +
Added qBittorent and bug fixing / performance increases for lighttpd
 +
 +
* www/filemanager.php: Slight optimization on the download file routines
 +
* etc/seedbox/config/template.lighttpd: Add stream configs, and define fastcgi min_procs, increase max_procs
 +
* lib/apps/packages.php: Remove avahi-daemon, install qbittorrent-nox
 +
* etc/seedbox/config/template.lighttpd: Added qbittorrent reverse proxy settings
 +
* util/configureLighttpd.php: Added qBittorrent config
 +
* util/userConfig.php: Added qBittorrent config
 +
* cron/checkQbittorrentInstances.php added
 +
* etc/seedbox/config/root.cron: Add check qBittorrent instances
 +
 +
==== 19/11/2021 ====
 +
 +
* scripts/cron/diskIostat.php: fix deb10 collected fields
 +
 +
==== 12/10/2021 ====
 +
 +
Log file is at ~/.rcloneLog
 +
Issue for sometimes getting 500 internal server error was because in some cases using ~/ did not function, where as at other times it works perfectly. Changed all of them to full paths using 'whoami'
 +
Did almost the same for deluge
 +
 +
* scripts/cron/checkRcloneInstances.php: Add logging, full path to htpasswd aswell
 +
* www/rclone.php: Add logging (comes on update, not on distribution package), full path to htpasswd aswell
 +
* www/deluge.php: Add logging, also comes via an central update not distribution package
 +
* scripts/cron/checkDelugeInstances.php: Add logging
 +
 +
==== 28/09/2021 ====
 +
 +
* scripts/cron/trafficLog.php: New grep rules had typo, which caused no effect
 +
* scripts/cron/trafficLog.php: Refactoring a little bit
 +
* scripts/cron/trafficLog.php: Additional checks and debugging
 +
* /scripts/lib/networkInfo.php: Default to 1000Mbps if couldn't determine otherwise
 +
* scripts/cron/trafficLog.php: Log if user 5min average exceeds 25% link maximum (so we can more easily find when the errors occur!)
 +
* scripts/cron/trafficLog.php: Additional grep rules for data collection, just in case. Identified that occasionally iptables throws an error, and that make throw us a complete curve ball
 +
* scripts/cron/trafficLog.php: Only try to reset OUTPUT chain stats. Identified miscounting is related on iptables failing to reset counters with invalid argumen on INPUT chain
 +
* scripts/util/makeMonitoringRules.php: Change initial mark
 +
* etc/seedbox/config/template.fireqos: Changed users to balanced priority
 +
* Syncthing distribution version changed to latest
 +
* scripts/lib/apps/syncthing.php: Check which version and update
 +
* scripts/lib/apps/rclone.php: Update to version v1.56.1
 +
* scripts/lib/apps/syncthing.php: Variable name type fixed
 +
* scripts/lib/apps/filebot.php: Updated to verion 4.9.4
 +
* scripts/util/userConfig.php: Put randomized rclone port (user can change this too)
 +
* scripts/util/configureLighttpd.php: Stop making so many backups
 +
* GUI tabs include rclone if enabled
 +
* GUI welcome page has rclone control buttons
 +
* GUI added rclone.php
 +
* etc/seedbox/config/template.lighttpd: Added rclone section
 +
* etc/seedbox/config/template.lighttpd: Increased per connection speeds
 +
* etc/seedbox/config/template.lighttpd: Disabled fcgi debug
 +
* scripts/util/configureLighttpd.php: Add rclone section
 +
* script/cron/checkRcloneInstances.php: Added
 +
* etc/seedbox/config/root.cron: Added check rclone instances every 2 minutes
 +
 +
==== 23/09/2021 ====
 +
 +
* scripts/cron/trafficLog.php: trim and format logged output values. On rare occasion there was extra line changes.
 +
* scripts/cron/trafficLog.php: Omit QOS MARK lines, this might've been occasionally the reason for miscalculation tho we cannot reproduce the issue
 +
* etc/seedbox/config/root.cron: Change traffic limit checks to once per hour, and traffic stats generation twice per hour
 +
* scripts/cron/trafficLimits.php: Silence error if trying to disable limit which is not enabled to begin with
 +
* scripts/cron/checkLighttpdInstances.php: Now also check with curl that we get 401 Unauthorized response.
 +
* scripts/cron/checkInstances.php: Check for rtorrent socket file now as well, if instance still found kill it.
 +
* scripts/util/update-step2.php: Add for deb10 apt update to get minor version to update, and remove apt-get dist-upgrade command
 +
 +
==== 22/08/2021 ====
 +
 +
* scripts/lib/apps/packages.php: Add python3-venv
 +
* scripts/util/setupLetsEncrypt.php: certbot changed again (EFF sure likes to break old users, don't they?), change over to venv/pip method. Since it's pip etc. let's hope it works for more than a week.
 +
* scripts/lib/apps/packages.php: Remove openvpn package
 +
* scripts/lib/apps/openvpn.php: disabled until support is fixed
 +
 +
==== 17/04/2021 ====
 +
 +
* scripts/lib/apps/packages.php: Fixed bug where some packages were not installed for Debian 10
 +
 +
==== 10/04/2021 ====
 +
 +
* scripts/util/setupLetsEncrypt.php: Changed around how it functions as certbot changed and the certbot auto script does not function anymore. Now certs should get installed again normally on Debian 10 systems. Debian 8 not supported at this point (untested)
 +
 +
==== 24/02/2021 ====
 +
 +
* scripts/cron/checkInstances.php: Comment out the pid file checking, not sure if this is necessary to date but may cause multiple executors issue
 +
* apps/packages.php: Add zsh and atop
 +
* scripts/cron/userTrackerCleaner.php: Check if there is existing instances, if there is do not launch
 +
* scripts/cron/checkInstances.php: Check against executor script instead of rtorrent instance itself. Fixes a issue where there might be hundreds of executor scripts running
 +
 +
=== Changes 2020 ===
 +
 +
==== 10/12/2020 ====
 +
 +
* lib/apps/deluge.php: Deb8 backwards compatibility fix
 +
 +
==== 26/11/2020 ====
 +
 +
* lib/apps/deluge.php: Some backwards compatibility fix with Deb8 repo package symlinks
 +
* lib/apps/packages.php: Deb10 compatibility fixes, add php-mbstring package
 +
* lib/apps/deluge.php: path typo fix for deb10 systems
 +
 +
==== 23/11/2020 ====
 +
 +
* lib/apps/deluge.php: Fixed compilation bug
 +
* util/setupUserHomePermissions.php: removed some of the rules
 +
 +
==== 12/11/2020 ====
 +
 +
* util/userConfig.php: Various fixes for deluge configuration + update nginx config and restart nginx
 +
* cron/userTrackerCleaner.php: Lower the number of checks even lower, just 2 users and max 20 torrents at a time
 +
* etc/seedbox/config/template.deluge.web.conf: Typo fix and base url fix
 +
 +
==== 10/11/2020 ====
 +
 +
* util/userConfig.php: Typo choosing deluge port fixed
 +
 +
==== 09/11/2020 ====
 +
 +
* lib/apps/deluge.php: Use repo version for non debian 10 installs
 +
* util/userConfig.php: Fix cache allocation, it is 16KiB blocks not MiBs
 +
 +
==== 08/11/2020 ====
 +
'''Tracker Cleaner enabled ++ Deluge Install'''
 +
 +
* cron/userTrackerCleaner.php: First filter list -- '''AND THIS IS NOW ENABLED'''.
 +
* lib/apps/rtorrent.php: If not Debian 10, try 0.9.6. This is part of debugging to see if 0.9.6 could be used again
 +
* lib/apps/deluge.php: Install deluge, in the most basic form no fail checks etc. hopefully works on all deb8/deb10 systems despite pythoness issues. There is some redundant commands here, but we'll fix them later
 +
* etc/seedbox/config/template.deluge*: Added base templates for Deluge
 +
* util/userConfig.php: Added base deluge configuration
 +
* etc/seedbox/config/template.nginx-user: Add deluge proxy settings for /deluge-USERNAME/
 +
* cron/checkLighttpdInstances.php: Logging said start rtorrent, fixed
 +
* cron/checkDelugeInstances.php: Start deluge instances if enabled via /home/USERNAME/.delugeEnable file existing
 +
* etc/seedbox/config/root.cron: Add checkDelugeInstances at 2 minute interval
 +
* +GUI update logging
 +
* +GUI deluge buttons + tab
 +
 +
==== 06/11/2020 ====
 +
 +
* seedbox/config/root.cron: check lighttpd instances once per minute
 +
* cron/checkLighttpdInstances.php: Check also php-cgi, and restart if php-cgi not present
 +
* seedbox/config/template.lighttpd: Back to more sensible fcgi parameters
 +
* var/www/error-502.html: Fixed the wait time numbers
 +
* cron/userTrackerCleaner.php: Cleaning of udp trackers from public torrents which block the whole rTorrent instance. This can be disable by creating .trackerCleanerDisable on user home directory
 +
* scripts/lib/devristo directory: library from https://github.com/Devristo/torrent/ to manage torrents
 +
* skel/.rtorrent.rc.custom: By default again create the .logs directory and log tracker notices, storage and socket critical errors
 +
* cron/checkLighttpdInstances.php: remove 2 sec delay between start events for each user
 +
* skel/.rtorrent.rc.custom: Turns out the logging levels are buggered too, only debug level shows any tracker action *sigh*
 +
* util/update-step2.php: Check for original empty rtorrent.rc.custom; and instead of deleting the old one, update with new one
 +
* seedbox/config/root.cron: added tracker cleaner, but that script doesn't yet do anything, we need to build the list first
 +
 +
==== 02/11/2020 ====
 +
 +
* util/update-step2.php: Remove the way too verbose logging added on .rtorrent.rc.custom
 +
* skel/.rtorrent.rc.custom: Comment out the logging stuff
 +
 +
==== 01/11/2020 ====
 +
'''Some major changes''': New rTorrent (latest git, udns branch) with some changes in how it is compiled. New traffic control mechanism via QoS since rTorrent limiting did not since update to 0.9.8
 +
 +
* lib/apps/rtorrent.php: Fix typo
 +
* lib/apps/rtorrent.php: More typoes fixed
 +
* lib/apps/rtorrent.php: rtorrent version checked here and target version variable to make updates easier
 +
* util/update-step2.php: remove checking rTorrent version as only the installer requires this
 +
* lib/apps/rtorrent.php: Add check if xmlrpc-c already installed, if not do not install again
 +
* util/setupNetwork.php: Clear all rules at start
 +
* util/makeMonitoringRules.php: Add connmark for users, for which no local network was matched for
 +
* etc/seedbox/config/network: Added basic network config to remove autodetect issues in future, default eth0 and 1Gbit speeds defined
 +
* util/update-step2.php: Notification to edit this file
 +
* util/update-step2.php: Moved various end of update stuff here, as /scripts/update.php should be as sparse as possible
 +
* update.php: Moved some stuff to update-step2
 +
* util/makeMonitoringRules.php: Added packet marking
 +
* util/setupNetwork.php: Added qos rule creation, and starts qos
 +
* cron/trafficLimits.php: Old timey rtorrent traffic limit removed, only puts limit info on user account now
 +
* util/setupNetwork.php: Does traffic limiting now on qos level
 +
* cron/trafficLog.php: typo fixed
 +
* lib/apps/rclone.php: well, path fix needed fixing :/
 +
* lib/networkInfo.php: changes to use the network config file
 +
 +
==== 30/10/2020 ====
 +
 +
* etc/seedbox/config/template.lighttpd: Change fastcgi settings to 1000 max requests.
 +
* etc/seedbox/config/template.lighttpd: Enable fastcgi debugging in hopes to catching why fastcgi crashes on some servers
 +
* scripts/lib/apps/packages.php: Added php7.3-xml php7-3.zip for debian 10 packages install
 +
* rTorrent / ruTorrent: Back to 0.9.8; udns branch did not solve any issues
 +
 +
==== 06/10/2020 ====
 +
 +
* skel/www/filemanager.php: Enforce https
 +
 +
==== 05/10/2020 ====
 +
 +
* etc/seedbox/config/template.lighttpd: Increase performance + stability, both PHP and throughput via altered settings. php-cgi with new rutorrent is crashing, increasing process count while decreasing max requests per children crashes will become less frequent likely.
 +
 +
==== 13/09/2020 ====
 +
 +
* cron/diskIostat.php: Debian 10 iostat mapping
 +
* www/rutorrent/plugins/hddquota/action.php: Update for debian 7, regexp modifiers changed
 +
* '''NEW FILEMANAGER!''' Using installed as "filemanager.php" and GUI updates to that if it's been installed.
 +
 +
==== 05/09/2020 ====
 +
 +
* util/userTransfer.php: Change the rsync spawn line authentication info, bug fix for Debian 10
 +
 +
==== 27/08/2020 ====
 +
This makes Debian 10 production ready with 2 caveats: No OpenVPN and Mega tools.
 +
 +
 +
* etc/seedbox/config/template.lighttpd: changed custom file from include_shell to just include (regular config file instead of shell script). New version of lighttpd fails to start unless a script is present, but empty file is ok
 +
* etc/seedbox/config/template.lighttpd: changed include_shell for create-mime to include wildcard as Debian 10 lighttpd has it renamed AND now fails to start if any file is missing from includes. this should work with debian 8 and debian 10
 +
 +
==== 05/08/2020 ====
 +
 +
* lib/apps/radarr.php: Bug in installer fixed, tried to get the installer package from wrong path. Updates to the latest version now on each server update as well.
 +
* util/userTransfer.php: Mistake in the rsync commandline fixed
 +
 +
==== 20/07/2020 ====
 +
 +
* .rtorrent.rc.custom: Added logging by default due to some regression with rtorrent lately -- users can freely disable this.
 +
* util/userTransfer.php: Added migration of rtorrent.rc.custom as well.
 +
* skel/www/rutorrent: ''Update to ruTorrent V3.10'' -- reintroduce check_port plugin as well as it seems to have received patches now. Include all standard plugins except diskspace and _cloudflare
 +
* lib/update.php: Some updates on rutorrent config.php configs
 +
* util/update-step2.php: Re-add updating rutorrent during this process
 +
* util/update-step2.php: Install Let's Encrypt cert during update
 +
* lib/update.php: Comment out rutorrent filemanager plugin stuff for now - retest at later date for addition
 +
 +
==== 16/07/2020 ====
 +
Ouch! As part of DDoS mitigation we started filtering other subnets as well, and we had blocked the network we utilize for OpenVPN connectivity. Removed that specific filtering as temporary measure.
 +
 +
* util/setupNetwork.php: Removed filtering 10.0.0.0/8
 +
 +
==== 15/07/2020 ====
 +
Core functionality for Debian 10 should work now, some applications need to still be checked but all core stuff should function as expected.
 +
 +
* lib/apps/packages.php: Install expect
 +
* util/update-step2.php: move rTorrent compilation to the lib/apps/rtorrent.php file and use that
 +
* lib/apps/rtorrent.php: Add this file + Install & compile xmlrpc-c r3116 (bumping version)
 +
* util/update-step2.php: Refactor ipset, iprange, firehol, filebot to lib/apps
 +
 +
==== 11/06/2020 ====
 +
Change rtorrent to latest git and libtorrent to the udns branch
 +
 +
* Install.sh: Comment out compiling libtorrent + rtorrent
 +
* lib/update-step2.php: bump to libtorrent udns branch
 +
* lib/update-step2.php: Delete check_port rutorrent plugin
 +
* rutorrent: remove check_port plugin from default plugins. This plugin causes a lot of confusion as it is broken most of the time.
 +
* rclone version bumped to 1.52.1
 +
 +
==== 27/03/2020 ====
 +
Basic debian 10 buster support. Experimental still, requires a lot of testing.
 +
 +
* listUsers.php -> fix interpreter path
 +
* addUser.php -> fix interpreter path
 +
* changePw.php -> fix interpreter path
 +
* filecopy.php -> fix interpreter path
 +
* suspend.php -> fix interpreter path
 +
* unsuspend.php -> fix interpreter path
 +
* recreateUser.php -> fix interpreter path
 +
* systemUsers.php -> fix interpreter path
 +
* cron/checkHttpd.php  -> fix interpreter path
 +
* util/populateUserDb.php -> fix interpreter path
 +
* util/setupUserHomePermissions.php -> fix interpreter path
 +
* install.sh: Debian 10 support and some cleanup of old deprecated stuff
 +
* util/checkUserHtpasswd.php: quiet error if lighttpd htpasswd file not found & commenting
 +
* lib/apps/packages.php: Debian10 support, some refactoring
 +
* lib/apps/mono.php: install package mono-complete, move deb8 mono install code from sonarr.php here
 +
* lib/apps/sonarr.php: Remove mono installation thing
 +
* lib/apps/radarr.php: Rudimentary radarr installer, but no config or testing yet
 +
* util/update-step2.php: Some refactoring, Debian 10 repos, commenting old deprecated code etc.
 +
 +
==== 23/03/2020 ====
 +
 +
* skel/www/rutorrent/plugins/cpuload: Removed. Forgot to remove earlier. It is extremely misleading even at best of times. It only shows processes running: https://github.com/Novik/ruTorrent/wiki/PluginCpuload
 +
 +
==== 21/02/2020 ====
 +
 +
* util/update-step2.php: Install tree, ncdu and weechat
 +
 +
==== 20/02/2020 ====
 +
 +
* util/update-step2.php: Install tmux
 +
 +
==== 07/02/2020 ====
 +
 +
* .rtorrentExecute.php: Add failsafe checks for rtorrent or executor already running, if more than 1x rtorrent running kill all rtorrent instances.
 +
* lib/rtorrentConfig.php: Remove rtorrent.rc custom option from here, it can be done via config
 +
* skel/.rtorrent.rc.custom: Added
 +
* rtorrent.rc template changed to try loading this file
 +
* rtorrent.rc template "modernized", and things moved around etc.
 +
* minimum peers lowered to just a few
 +
* template.nginx-conf: keepalive timeout 65s -> 20s, client max body size 4096M -> 8192M, TLSv1 removed, SSLv3 added
 +
* template.nginx-site-default: HTTPS/SSL Section separate to template.nginx-site-default-default-ssl, replace with placeholder which will cause nginx not to launch unless configured by configurator
 +
* template.nginx-site-default-ssl: added with original config, except protocols changed and ciphers matching let's encrypt
 +
* util/createNginxConfig: make building https/ssl config around either self signed cert, or let's encrypt
 +
* util/setupLetsEncrypt.php: A quick let's encrypt cert install script. Pass e-mail address and all should function
 +
 +
==== 06/02/2020 ====
 +
 +
* lib/rtorrentConfig.php: Add support for .rtorrent.rc.custom - this is included on top of the config file, so resource settings should get overwritten and remain same
 +
 +
==== 02/02/2020 ====
 +
Bonding interface support
 +
 +
* lib/networkInfo.php: Network information fetching snippet
 +
* util/setupNetwork.php: Refactor link determination stuff to networkInfo.php. Fix static definitions for eth0
 +
* lib/apps/vnstat.php: Use networkInfo output for link + speeds
 +
* cron/trafficLog.php: Use linkspeed to determine maximum usage.
 +
 +
=== Changes 2019 ===
 +
 +
==== 08/12/2019 ====
 +
 +
More rTorrent regression fixes for 0.9.8 -> simple renamings of old config options etc.
 +
 +
* lib/rTorrentConfig.php: Use new template with these fixed, and for memory instead of bytes define as megabytes
 +
 +
==== 06/12/2019 ====
 +
 +
rTorrent regressions fix for 0.9.8, described at: https://github.com/Novik/ruTorrent/commit/ec8d8f1887af57793a671258072b59193a5d8d6c
 +
 +
* change skel according to above patch
 +
* util/update-step2.php: update above user files
 +
 +
==== 05/12/2019 ====
 +
* Remove use_udp_trackers and extranous use_udp_set from .rtorrent.rc template, and bump date.
 +
* lib/rtorrentConfig.php: Use the new version
 +
* util/update-step2.php: '''Bump rtorrent version to 0.9.8 libtorrent to 0.13.8'''
 +
* util/update-step2.php: Remove use_udp_set = yes from local template AND from users
 +
* install.sh: Bump rtorrent version to 0.9.7 libtorrent to 0.13.8
 +
* util/update-step2.php: Fix updating repos and add dist-upgrade -y
 +
* etc/seedbox/config/template.openvpn.server.config: change to use aes-256-cbc cipher
 +
* lib/apps/openvpn.php: Update server config if template has changed
 +
* lib/apps/openvpn.php: Fixed long standing bug with config tar package not being created properly (empty)
 +
* etc/seedbox/config/root.cron: checkDirectories changed to hourly. Add @reboot tasks for checkInstances, checkLighttpdInstances and checkDirectories and have some delays for the @reboot tasks so not everything launches on the same second.
 +
* soft.sh: do not delete scripts directory - usually nothing is removed from here, just added or updated. Removed web proxy removal as we have not had that many years in any case
 +
* util/setupNetwork.php: Added logging tcpsack
 +
* util/update-step2.php: Put date into /var/run/pmss/updated for reference, also make sure dir exists.
 +
* cron/checkGui.php: New cronjob, every 10 mins to check if index.php is intact, copy new one if not. This can occasionally happen when user is over quota and beyond burst period.
 +
* etc/skel/www: index.php, info.php, stats.php, welcome.php updated to the latest being distributed.
 +
 +
==== 29/11/2019 ====
 +
* util/setupNetwork.php: Add iptables rule to drop certain size mss, mitigating tcpsack attacks; and further disable tcp_sack kernel
 +
* util/setupNetwork.php: Fix not clearing INPUT rules first, thus making multiples of the same rules.
 +
* util/setupNetwork.php: Dropping specific targets now use insert (to make high on list) and accept rules at bottom
 +
* root.cron: Long standing bug with @reboot setupNetwork cron job fixed. This was causing why OpenVPN does not always function as expected.
 +
 +
==== 21/11/2019 ====
 +
* ruTorrent: remove diskspace plugin, and include our own disk quota plugin
 +
 +
==== 08/11/2019 ====
 +
* cron/trafficLimits.php: Add unix socket support
 +
* cron/trafficLimits.php: Check & create runtime directory if does not exist
 +
* cron/trafficLimits.php: Issue second traffic limit removal command to make sure it actually happens
 +
* ~/.rtorrentExecute.php: Drop waiting periods to make restart happen faster
 +
* root.cron: Drop checkinstance periods from 5 minutes to 2 minutes for faster restarts
 +
* root.cron: Redirect checkDirectories log output to /dev/null. Cron may not execute if redirected to log directory and log directory does not execute ... making circular issue
 +
* root.cron: Drop traffic stats collection from 5 mins to 10mins
 +
* cron/trafficStats.php: Save daily traffic consumption to ~/.trafficData
 +
* www/info.php: Add chartjs loading
 +
* www/stats.php: Add daily charts for traffic consumption
 +
* util/update-step2.php: Install sox, nzbget
 +
* util/userConfig.php: Removing some commented out code
 +
* RuTorrent updated to latest V3.9 stable release! Not updating existing users yet, only for userspace rebuilds & new users
 +
 +
==== 06/10/2019 ====
 +
* config/root.cron: Typoes in the renice commands fixed.
 +
 +
==== 28/08/2019 ====
 +
* lib/rtorrentConfig.php: added file exists check and creation in the write config function, in case user has deleted their config file
 +
 +
==== 06/08/2019 ====
 +
* cron/trafficStats.php: Check runtime directory readability and change way of creating to via shell
 +
* fix localnet filepermissions on rtorrent conf
 +
* change rtorrent config template scgi settings
 +
* rtorrent config chmods localnet file to 644 to avoid random permission issues
 +
* change rutorrent config scgi settings
 +
 +
==== 30/04/2019 ====
 +
* apps/packages.php had a typo this has been fixed.
 +
* apps/rclone.php: bug fixed version checking, every other update would remove rclone completely etc.
 +
 +
==== 29/04/2019 ====
 +
* Change jessie repos again, and disable valid until verification for repos to continue functioning as expected
 +
* refactored a lot of apt package install to lib/apps/packages.php
 +
* refactored a lot of package install from install.sh to lib/apps/packages.php
 +
* refactored more python stuff to lib/apps/python.php
 +
* Some update-step2 refactoring on useless legacy stuff, remove api remotekey etc.
 +
* Refactored runtime directories checking from update-step2 as a new cronjob - further found potential security issue where a local user could after reboot gain priviledges to the runtime data in /var/log/pmss or logs /var/log/pmss
 +
 +
==== 23/04/2019 ====
 +
* Update jessie repos
 +
* Rework sonarr/nzbdrone installer, staging due to jessie repo changes was a necessity
 +
* create /scripts/lib/apps folder to refactor different application installers for easier maintenance
 +
* lib/apps: did megatools, mono, sonarr installers, btsync
 +
* Added resilio sync '''THIS IS UNTESTED 100%'''
 +
* Renamed btsync to be btsync1.4 and created symlink btsync to point to 2.2 '''WARNING: If you are running btsync 1.4 this might break you'''
 +
* cron/trafficStats.php: Make sure runtime temp directory exists. Sometimes after reboot this folder might be missing for few hours
 +
* update.php: Remove apt-get update, move it to update-step2. comment out setupApiKey as that is not currently being utilized
 +
* update-step2.php: Remove debian lenny and squeeze repos etc. as those have not been used for years in production. There might still be some wheezy installations in the wild so not touching those.
 +
* update-step2.php: For some bizarre reason there was apt-get install for rar, unrar and mktorrent in the repo configs, removed those but made sure those are installed quickly after
 +
* lib/apps/syncthing.php: Syncthing installer '''THIS IS UNTESTED 100%''' it may or may not work. Probably does work.
 +
* lib/apps/openvpn.php: Move it here, some refactoring
 +
* lib/apps/rclone.php: Move installation here, some refactoring and commenting for easier future updates. ''Version updated to 1.47'' let us know if there is regressions due to update
 +
* lib/apps/vnstat.php: Move install + config here, move various spread around bits regarding deb8 conflicts to here as well.
 +
* update-step2.php: More commenting and deb version peculiarity fix refactoring and removing more support bits for deb5+6
 +
* lib/apps/python.php: Beginnings of this nightmare. Only added youtube_dl pip3 install so far.
 +
* util/makeMonitoringRules + cron/trafficLog: localnet file was using wrong path, fixed it to be /etc/seedbox/config/localnet
 +
* update-step2: move above mentioned localnet file during update
 +
* setupSkelPermissions: set localnet file permission to globally readable - yes i know wrong place but there was some openvpn configs already here too
 +
* lib/rtorrentConfig.php: Add localnet preferred ipv4 filtering if file is present
 +
 +
==== 04/01/2019 ====
 +
* util/update-step2.php: Install spidermonkey JS engine for pyload use, packages
 +
 +
=== Changes 2018 ===
 +
==== 11/12/2018 ====
 +
* seedbox/config/root.cron: Add ionice classification for cp as idle process
 +
* seedbox/config/root.cron: Add renice & ionice classification for ffmpeg and rclone as lowest priority
 +
* seedbox/config/root.cron: Ever so slightly bump up the ionice classification for rtorrent processes
 +
* seedbox/config/root.cron: Increase process checking frequencies for faster restarts etc.
 +
* util/update-step2.php: Added package 'ranger' to be installed, a ncurses frontend filemanager.
 +
* util/update-step2.php: Install nethack ... Because nethack! ;)
 +
 +
 +
==== 02/10/2018 ====
 +
* util/setupSkelPermissions.php: Also sets openvpn config dir perms. TODO refactor, rename etc.
 +
* util/update-step2.php: put openvpn config to the file update list
 +
* util/update-step2.php: change filemanger recycle bin permissions to 771
 +
* util/update-step2.php: re-enable creating openvpn-config.tgz when making the openvpn config the first time
 +
 +
==== 21/09/2018 ====
 +
* util/update-step2.php: Install Filebot 4.8.2
 +
 +
==== 17/09/2018 ====
 +
* Release was actually a dev version and latest updates were not pushed, this has been fixed. Caused myriad of network accounting issues etc. weird ones.
 +
 +
==== 10/09/2018 ====
 +
* proftpdTemplate: Add directive PassivePorts 60000-65535 to always have a known range
 +
 +
==== 06/09/2018 ====
 +
* util/setupNetwork.php: Bogon/Martian drop filters. Base groundwork to add a global bad actor banning mechanism (ie. bruteforce attempts)
 +
* util/update-step2.php: Install package ipset
 +
* util/update-step2.php: Compile & install iprange
 +
* util/update-step2.php: Compile & install firehol
 +
* etc/seedbox/config/template.nginx-user: Increase limit rates to: 500MiB full speed, after that 4096KiBps for users. Public remains as low as before.
 +
* etc/seedbox/config/template.nginx-proxy_params: buffering disable
 +
* etc/seedbox/config/template.lighttpd: Increase server max rate to 51200KiBps, and per connection to 16386KiBps, thus allowing nice bursting at beginning of a transfer :)
 +
* util/update-step2.php: remove some very ancient resolver updates
 +
* util/update-step2.php: remove a deprecated testfile removal
 +
* util/setupNetwork.php: Updated link speed parsing
 +
* util/setupNetwork.php: Determine our net link for getting speed with ethtool
 +
 +
==== 03/09/2018 ====
 +
* util/makeMonitoringRules.php: Trim localnets file
 +
* cron/trafficLog.php: Trim localnets file
 +
* cron/trafficLog.php: Fix grep clauses and a few other things related to the localnets monitoring
 +
* showTraffic: Remove each line "Traffic", kinda redundant. making it easier to read :)
 +
* cron/trafficStats.php: Logging date typo fix, even heavier parse error message suppression.
 +
* config/root.cron: Increase trafficStats cron to every 5 minutes as well, and wait 20secs for logging to complete
 +
* cron/trafficLog.php: More grep clause fixes, data collection fixes etc. in relation to local networks. Was not saving general data usage.
 +
* showTraffic: Change when to go for larger measurement unit for display purposes.
 +
* lib/traffic.php: Save local data consumption as well.
 +
* util/remote/userTraffic.php: Change output to support "normal" and "local" data consumption metrics
 +
 +
==== 02/09/2018 ====
 +
* showTraffic: make username field wider and shorten if -local file being displayed.
 +
* showTraffic: some display output formatting fixes
 +
* cron/trafficStats.php: notice of parse error supression was in wrong place
 +
* cron/trafficLog.php: Change to 1250MB/s max network utilization for 5minute average to enable monitoring. Still #TODO use ethtool to determine this
 +
* cron/trafficLog.php: log local networks separately on -localnet files
 +
* cron/trafficLog.php: Bug fix unmatched traffic metering (wrong file)
 +
* util/makeMonitoringRules.php: Separate local networks traffic monitoring per user, and finally add a "unmatched owner" rule for local networks
 +
* util/makeMonitoringRules.php: formatting fixes
 +
* New optional config file: /etc/seedbox/localnet, you can input your local networks here with same annotation as iptables. One per line
 +
 +
==== 20/08/2018 ====
 +
* util/update-step2.php: Some typo fixes
 +
* util/userTransfer.php: Add public web directory and homedir rutorrent settings folder into migration
 +
* util/update-step2.php: Add package python-virtualenv to be installed
 +
 +
==== 18/08/2018 ====
 +
* util/update-step2.php: Install gdrivefs, or atleast try ...
 +
* util/update-step2.php: remove python-pip apt package, easy_install it and link properly. This hopefully fixes python pip issues more than creates new issues, but the whole repo is horribly horribly broken with constantly stuff breaking.
 +
* util/update-step2.php: Add unionfs-fuse, sshfs and s3fs packages.
 +
 +
==== 12/08/2018 ====
 +
* util/update-step2.php: Fix flexget installation
 +
 +
==== 11/08/2018 ====
 +
* util/ftpConfig.php: Template file changed for 16 connectios per user & host max, plus max instances raised to 60 from 30. No changes to that file, just the template
 +
 +
==== 10/07/2018 ====
 +
* util/update-step2.php: Updated rclone to version 1.42. Old download link was not functioning anymore, so moved hosting to our own server.
 +
 +
==== 11/06/2018 ====
 +
* util/userTransfer.php: Disable compression for transfers. Most data is anyways compressed, so this ensures higher data rates.
 +
* util/update-step2.php: Add installation of genisoimage xorriso
 +
 +
=== Changes 2017 ===
 +
 +
 +
==== 28/07/2017 ====
 +
* cron/trafficStats.php: Fix date formatting for output
 +
* cron/trafficStats.php: Limit parse error messages + make supression if too many (50 for single user)
 +
 +
==== 26/07/2017 ====
 +
* rclone update to v1.37
 +
* New rtorrent.rc template file, with some new options enabled
 +
 +
==== 19/07/2017 ====
 +
* Rename template.nginx to template.nginx-sites-default
 +
* Add template.nginx-conf for /etc/nginx/nginx.conf
 +
* template.nginx-conf has client_max_body_size 4096M -> Fixes the 1MB file upload size limit which crept back in earlier
 +
* Fix update-step2.php a debian version selector, for which the earlier debug code was for
 +
* Update util/createNginxConfig.php to match above templating changes
 +
 +
==== 18/07/2017 ====
 +
* Added packages htop mtr aria2 to be installed by default
 +
* Deb8 specific package setups regarding git and znc debug + extra apt-get instructions to attempt get git installing finally
 +
* Add also systemctl instruction for lighttpd not to start
 +
 +
==== 12/07/2017 ====
 +
* Yet another vnstat fix (permissions)
 +
* Fix rclone install path
 +
 +
==== 13/06/2017 ====
 +
* cron/cpuStat.php: Collect CPU usage stats, user, nice, sys and idle for 2mins. Saves to /var/run and http publicly fetchable serialized output
 +
* config/root.cron: Add cpuStat to run every 15mins, after 45sec delay (avoid other scheduled cronjobs this way)
 +
 +
==== 25/03/2017 ====
 +
* util/update-step2.php: Hack for nzbdrone & mono to get them installed on a Debian 8 system
 +
* util/update-step2.php: Deb8: get prepackaged easy-rsa and configure openvpn, make path issue symlinks to avoid deb8 issues
 +
* util/update-step2.php: revert back to deb7 specific openvpn config to be the normal case and add selection for differences between deb8/deb7.
 +
* util/update-step2.php: remove deb8 speficic openvpn rsa config
 +
* rtorrent.rc min peers setting increased slightly
 +
* lib/rtorrentConfig.php: revamp port selection method
 +
 +
==== 11/03/2017 ====
 +
* util/update-step2.php: Refactoring on git, znc, python3, acd_cli installation (deb8 specific) in an attempt to figure out Deb8 issues regarding these (part installed, part not!)
 +
* util/update-step2.php: added lighttpd to list of daemons not to be run as system (only per user). TODO still is to check runlevel management changes for Deb8 (many repo pkgs has not been updated neither)
 +
 +
==== 07/01/2017 ====
 +
* cron/cgroup.php: Do not execute currently, not complete
 +
* cron/diskIostat.php: Some commenting and slight refactor
 +
* cron/trafficStats.php: refactor, commenting, couple typo fixes
 +
* cron/checkInstances.php: Slight refactoring, some commeting and clarifying
 +
* cron/checkQuotas.php: remove API calls, that method is deprecated
 +
* cron/trafficLog.php: adding a few code comments, increase temp file entropy
 +
 +
 +
=== Changes 2016 ===
 +
 +
==== 29/12/2016 ====
 +
* skel/rutorrent/plugins/autools/plugin.info: add to info note for watch dir breaking things
 +
* plugins/autotools/lang: added warnings to en, de, fi, es
 +
* install.sh: add zip pkg
 +
* install.sh: add -j make parameter
 +
* util/update-step2.php: add -j make parameter
 +
* util/update-step2.php: add zip pkg
 +
* addUser.php: comment out retracker stuff
 +
 +
==== 07/12/2016 ====
 +
* util/update-step2.php: (Re)install git, some updates on pkg management on deb8
 +
* util/update-step2.php: Fix vnstat not updating on deb8
 +
* util/update-step2.php: Install Acdcli for amazon cloud drive access
 +
* cron/diskIostat.php: fail if no serial block devices found
 +
* cron/diskSmart.php: Added to collect smart data of disk drives
 +
 +
==== 27/10/2016 ====
 +
* cron/diskIostat.php: Include server timestamp on the results
 +
* config/root.cron: Collect IOStat results every 5mins instead of 15mins (2min average)
 +
 +
==== 01/10/2016 ====
 +
* util/update-step2.php: Add pigz (parallel implementation of gzip, [http://zlib.net/pigz/]) to be installed
 +
* util/update-step2.php: Small refactoring on apt package installs
 +
 +
==== 25/09/2016 ====
 +
 +
* lib/update.php: ruTorrent config add setting top directory correctly, filemanager plugin for example requires this
 +
 +
==== 08/09/2016 ====
 +
* util/update-step2.php: Add Jessie (Deb8) repors
 +
* util/update-step2.php: Add OpenVPN config for Jessie, OpenVPN config now distro specific
 +
* util/update-step2.php: Remove some old (deb5/6) apt-get instructions
 +
* util/update-step2.php: Debian 8 ffmpeg install from backports added
 +
* util/update-step2.php: znc package selection for Deb8
 +
 +
 +
==== 05/09/2016 ====
 +
Preliminary support for deb8, most things work.
 +
* install.sh: Lots small changes to add support for deb8
 +
* util/update-step2.php: Added headers to launch from cli
 +
 +
==== 03/09/2016 ====
 +
* util/update-step2.php: Install funcsigs pip package, slight update to upgrade list of packages for flexget
 +
 +
==== 03/08/2016 ====
 +
* Small visual updates
 +
 +
==== 25/07/2016 ====
 +
* util/update-step2.php: Removed some apt-get instructions as they are in install.sh already
 +
* install.sh: Some slight refactoring on the package installs
 +
* util/update-step2.php: Install rclone
 +
* cron/diskIostat.php: Make iostat current stats available also via direct http download
 +
 +
==== 21/02/2016 ====
 +
* checkInstances.php: Typo fix
 +
* checkLighttpdInstances.php: Typo fix
 +
 +
==== 07/02/2016 ====
 +
* terminateUser: Comments, multiple killall and userdel directives as procs sometimes just don't want to die ... or are being relaunced. TODO check why this actually happens instead of this hack
 +
* cron/checkInstances.php: If user is suspended, skip trying to start rTorrent ... Infact kill all procs :)
 +
* cron/checkLighttpdInstances.php: Same thing here
 +
 +
==== 28/01/2016 ====
 +
* cron/trafficLimits: Some refactoring, adjusting limits for new speed caps and added code to choose limit level based on total traffic limit as well (512K/s allowed to do more than limit on minimal plans)
 +
* lib/rtorrentXmlrpc: Add function for setting download rate
 +
* cron/trafficLimits: Restrict download speed as well IF entry plan (less than 500GiB traffic limit) to 20Mbps
 +
 +
==== 24/01/2016 ====
 +
* update-step2: Install libffi-dev, python-dev, pyopenssl ndg-httpsclient pyasn1 cryptography for better flexget support
 +
* update-step2: replace some nameservers with sed to OpenDNS (Online.net is censoring some trackers)
 +
 +
==== 14/01/2016 ====
 +
* update-step2: Removed creating /etc/autodl.cfg, instead removing it, as it's not a prerequisite for autodl anymore and we can allow all permissions.
 +
 +
==== 11/01/2016 ====
 +
* addUser.php: Set also user homedir .trafficLimit file (so we can display it in GUI), slight refactor on this section
 +
* util/userTrafficLimit.php: Same thing as above
 +
* (GUI) Stats: Added traffic limit number
 +
* (GUI) Welcome page: Added meter for traffic limit
 +
 +
==== 05/01/2016 ====
 +
* util/update-step2: Initially typoed filename as tesfile, remove that and create the correct file
 +
* util/update-step2: Create /var/www/testfile, 100M random data
 +
 +
==== 01/01/2016 ====
 +
* cron/diskIostat: Added history logging raw output (We are seeing IOPS outputs of 1000+ from magnetic drives, so need debug data for that)
 +
* cron/diskIostat: History logging wrong output mode (should've been append).
 +
 +
=== Changes 2015 ===
 +
 +
==== 30/12/2015 ====
 +
* update-step2: Refactoring the openvpn setup section, which also bug fixes the situation where openvpn is setup but configs for users were not made
 +
 +
==== 28/12/2015 ====
 +
* upate-step2: Bug fix: cgroup-bin apt-get did not have autoconfirm, requiring human intervention to complete update
 +
 +
==== 26/12/2015 ====
 +
* cron/trafficLog: Log www-data & unmatched traffic as well
 +
* cron/trafficStats: Parse for all log files found on logging dir, so includes unmatched, www-data, terminated users etc.
 +
* cron/trafficStats: Log total data for the month period
 +
* cron/trafficStats: Removing legacy code, small refactoring and commenting bits
 +
* showTraffic: Show for all data files found
 +
* lib/traffic.php: saveUserTraffic now saves to /var/run/pmss/trafficStats, and creates that directory if it does not exist
 +
* lib/traffic.php: Bug fix: Skipped if data exceeded 7500Mb in 5mins... that's only 25MB/s. Now limit is 500MB/s with 150 000MiB check.
 +
* showTraffic: Now uses /var data files. Instead of skipping on missing, throw an error
 +
* showTraffic: If data is 0 for the month, skip displaying this user (gone already)
 +
* util/makeMonitoringRules.php: Add 'www-data' to list of users
 +
* showTraffic: Display month total across all users in TiB
 +
* util/update-step2: Fix vnstat configuration typo to set the maximum link speed to 1Gbps
 +
* cron/cgroup.php: Created, initial version. Needs quite a bit of refactoring, testing level right now.
 +
* util/update-step2: install cgroup-bin, add fstab cgroup entry and mount it
 +
* etc/seedbox/config/root.cron: Add cgroup cron job, execute once per hour
  
−
==== 14/10/2012 ====
 
−
* pulsedBox authentication pruning to filter/fix the most common user errors
 
  
−
==== 22/09/2012 ====
 
−
* Fixed quota parsin in GUI when using LVM device /dev/mapper/ type addressing
 
−
* Updated installer to install rTorrent 0.9.2 and libTorrent 0.13.2
 
−
* Fixed updater setup of squeeze repos
 
−
* ruTorrent update, does include saving user preferences etc. to the new one
 
−
* Fixed a bug relating to .pulsedmedia mgmt features, incorrect order for permissions setting
 
−
* Update-step2 mild refactoring
 
−
* Removed useless apt-get install from update.php
 
−
* Added ffmpeg along with screenshots plugin
 
−
* Removed unpack plugin (way too high resource consumption)
 
−
* getfile plugin included now
 
−
* Applied quota parsing fixed to hddquota plugin and made slightly more dynamic
 
−
* Lowered time before displaying welcome page quota information
 
  
−
==== 17/09/2012 ====
+
==== 21/12/2015 ====
−
* Removed proxy from soft.sh as this hasn't been included for sometime now.
+
* update-step2: run apt-get upgrade at the end
−
* Updated rTorrent to 0.9.2 and libTorrent to 0.13.2 official releases
 
  
−
==== 10/06/2012 ====
+
==== 20/12/2015 ====
−
* PEX + DHT enabled by default. DHT port same as SCGI port for now. Template changed - not the actual code, so in all versions the default has changed. Server specific configuration can disable.
+
* update-step2: Install btsync 2.2, as btsync2.2. System will have 1.4 as ''btsync'' and 2.2 as ''btsync2.2''
  
−
==== 22/05/2012 ====
+
==== 19/12/2015 ====
−
* Quota display bugs fixed: Servers with UUID assignment instead of device and in some cases collecting the total quota limit had last number cut off
+
* update-step2: separate setup apt sources.list.d/sonarr.list. This also means Sonarr will now update normally.
 +
* createNginxConfig.php: Now always run lighttpd config. This is a bug fix, where manually changed lighttpd port would eventually mismatch on nginx
 +
* configureLighttpd.php: Increased potential port range.
 +
* skel/.lighttpd/custom: Empty file created to avoid warnings. This is where users can input their custom lighttpd settings if they need some.
 +
* userTransfer.php: Added .lighttpd/custom for migration
 +
* userTransfer.php: Decreased the time for re-attempting user migration
 +
* recreateUser.php: Also copy htpasswd file from old version
  
−
==== 24/04/2012 ====
+
==== 08/12/2015 ====
−
* Munin configuration updates for .pulsedmedia.com
+
* (GUI) Change welcome page quota display to use Kibibyte format to avoid confusion.
−
* Updated method of checking ruTorrent version to avoid "updates" when none is needed
 
−
* More conditions for user space update, if removed user but something remains as backup / as a failure of user termination script
 
−
* Refactored a bit update-step2 fixing when .rtorrent.rc is updated
 
  
−
==== 06/04/2012 ====
+
==== 30/11/2015 ====
−
* Compile rTorrent always when version mismatch during update, even if empty/not installed
+
A privileged user could use an internet download manager to basicly DOS the server with huge amount of threads trying to download large files (gigs in size), with sufficient thread quantity eventually linux kernel couldn't schedule any I/O time for disk nor ram for essential applications such as SSH. This could not be done by unprivileged user since requirement is very large file sizes. This would lead to a race condition as nginx wants to cache as fast as possible, hence "downloading" from user lighttpd process as fast as possible, which results in heavy RAM consumption and 'especially' heavy disk I/O activity - with sufficient threads scheduling breaks down.
−
* Added download speed limit when downloading via ajaxManager
 
−
* Changed chat tab to use username instead of random
 
  
−
==== 27/03/2012 ====
+
* template.lighttpd: Introduce per connection rate limit (2048k/s) and global rate limit (12800k/s)
−
* Update debian 5 repos to the archive ones
+
* template.nginx-user: Introduce user con limit to 16, rate limit (2048k/s) after 100MiB
−
* On install also enable fastcgi-php (Squeeze)
+
* template.nginx-user: Introduce public con limit to 8, rate limit (512k/s) after 100MiB
−
* Get quota from apt repo, and do not compile on install (Lenny support is now limited)
+
* tmeplate.nginx: Add shared memory zone for connection limits
−
* Updated squeeze repos
 
−
* Slightly refactored update-step2 and install
 
  
−
==== 21/03/2012 ====
+
==== 18/11/2015 ====
−
* Removed autotools + cpuload plugin from skel rutorrent
+
* util/userConfig.php: Lower limit on filesLimit for very small disk quotas, set to 15000
 +
* (GUI) welcome.php: Display of /etc/seedbox/config/vendorWelcome moved to top of the welcome page
  
−
==== 20/03/2012 ====
+
==== 12/11/2015 ====
−
* rTorrent + libtorrent version to be changed to GIT 2011/11/08 which is slightly updated from release 0.8.9, few bug fixes etc. GIT information retained if you want to check validity.
+
* cron: renice mono and ionice mktorrent added
−
* Removal of cpuload + autotools rutorrent plugins. CPULoad being misleading at best, autotools haven't worked for ages and developer is MIA.
+
* update-step2: install some perl modules
−
* Changed fetching libmediainfo packages from pulsedmedia.com instead of sourceforge.
 
−
* Choosing libmedia info + libs based on debian version. Now Debian 6.0 should be fully supported in updates.
 
−
* Creating symlink /usr/local/bin/mktorrent -> /usr/bin/mktorrent  thus fixing ruTorrent torrent creation in some Debian 6.0 systems (with updated ruTorrent+rTorrent)
 
−
* Cleaning installed libtorrent
 
  
−
==== 04/03/2012 ====
+
==== 26/10/2015 ====
−
* Allow key based authentication on SSH
+
* .lighttpd.conf: Add include_shell for .lighttpd/custom for user custom settings
−
* Pulsed Media specific management code - not added unless server is for .pulsedmedia.com
+
* .lighttpd.conf: Add rutorrent no cache headers, and also add mod_setenv to be loaded
−
* Added srvmgmt to filtered users --- NOTICE: srvadmin user account will be deprecated in a future release
+
* rtorrent.rc template: Increased base buffers
  
−
==== 12/02/2012 ====
+
==== 24/10/2015 ====
−
* added scripts inc for files to be updated/copied on update
+
* rutorrent/plugins/hddquota: Bumped version to 1.15
 +
* ruTorrent: '''Updated to version 3.7'''
 +
* install.sh: Bumped rTorrent/libTorrent to 0.9.6/0.13.6, bumped xmlrpc-c to revision 2776
 +
* rTorrent: '''Updated to 0.9.6/0.13.6'''
  
−
==== 10/02/2012 ====
+
==== 16/10/2015 ====
−
* Fixed torrent creation from ruTorrent
+
* update-step2: Make openvpn config package even if this is not an install run
−
* Fixed quota plugin (JSON parser has been changed to strict parsing)
+
* update-step2: fix flexget install if clause
−
* Updated quota plugin version information
 
−
* Fixed ajax manager recycle bin
 
−
* Made ruTorrent to update browser cache
 
−
* Created framework for easy user space files updating during server update (files dependant upon server upgrade)
 
−
* Removed recreateLighttpdConfig + lighttpd restart from update step 2 (should by now be in main update in 99.9%+ servers)
 
−
* Changed update.php to do lighttpd force-reload instead of restart
 
−
* Refactored update-step2 slightly: Moved specific files, and user specific stuff into single loop
 
−
* Added several packages to be installed during server installation, which were only on update. They still remain on update as well before better solution can be done (dselect)
 
−
* Filtering backup user dirs out from user list
 
−
* During scripts update, also update system packages
 
  
−
==== 09/02/2012 ====
+
==== 12/10/2015 ====
−
* Reversing yesterday's update due to security concerns
+
* pulsedBox: Login for mds now fixed
−
* Removed installer speedups due to stability concerns
+
* ruTorrent: Changed URI for css & js files to make sure cached copies are not attempted with new version
 +
* update-step2: put back in to update ruTorrent index.html
 +
* update-step2: Configure OpenVPN, Create certs, create client config package and provide files to users at /home/openvpn-**servername**.ovpn and /home/openvpn-**servername**.crt where as dots (.) have been replaced with dashes (-)
 +
* update-step2: Make a tarball of OpenVPN config in skel, and provide it to userspace as well.
 +
* setupNetwork.php: Create OpenVPN masquerade rules
 +
* (GUI) welcome.php: Add beta disclaimer to ownCloud
 +
* update-step2: Oops, openvpn config should probably reside under www
 +
* update-step2: trim hostname before using for filename
 +
* update-step2: flexget installation string match fix
  
−
==== 08/02/2012 ====
 
−
* Fixed typo in install script preventing quota from being compiled
 
  
−
==== 05/02/2012 ====
+
==== 11/10/2015 ====
−
* Lighttpd session ''"stability"'' upgrade, fixing issue causing session timeouts/re-requiring authentication
+
* update-step2: Install mkvtoolnix, add mkvtoolnix repo
 +
* userConfig: Refactoring on ruTorrent stuff
 +
* update-step2: Refactoring on ruTorrent stuff
 +
* skel/www/rutorrent: Updated to latest with new plugins
 +
* etc/seedbox/config/template.rutorrent.config: Updated to new
 +
* etc/seedbox/config/template.rutorrent.access: Updated to new
 +
* lib/update: Add rutorrent config function
 +
* userConfig: Change to using that function
 +
* update-step2: Reconfigure ruTorrent
 +
* update-step2: Reordering all together everything related to ruTorrent on per user basis and some refactoring
 +
* '''ruTorrent Update''' Latest version, should migrate settings etc. and reconfig properly :)
 +
* update-step2: removal of removing check port plugin. Seems to be working ok again?
  
−
==== 04/02/2012 ====
+
==== 10/10/2015 ====
−
* Added mediainfo
+
* update-step2: Install flexget, buildtools32, python-pip
−
* Added server root document to redirect to pulsedmedia.com
 
−
* Added php5-geoip extension to be installed
 
−
* Simplified update-step2 scripts being installed :)
 
−
* Removed apt dist-upgrade from install
 
−
* Simplified (less) apt-get lines during install
 
−
* Reordered apt-get commands during install for swifter installation
 
−
* Removed some "junk" code from install
 
−
* Updated copyright text in install.sh
 
−
* Added some information on header of install.sh
 
−
* Install.sh some additional comments
 
−
* Made installation slightly quicker by usage of background processes (BETA, if your disk is extremely slow this may cause installation to fail)
 
−
* Reordering other items in install.sh to make installation faster
 
−
* Some preliminary work for supporting whitelabeling
 
−
* Removed umask, hash interval, hash max tries rtorrent configures (deprecated)
 
−
* '''Updated rTorrent + ruTorrent to versions 0.9.0/0.13.0'''
 
−
''Note:'' Please report any issues due to the upgrade of rTorrent immediately to Pulsed Media. The patch has been tested on a small group of servers, but all situations are hard to foresee.
 
  
−
==== 31/01/2012 ====
+
==== 09/10/2015 ====
−
* Fixed some typos in update-step2
+
* install.sh: Slight reordering of package installs to make initial setup even faster (human time wise)
 +
* install.sh: Add lftp to be installed
 +
* update-step2: Add lftp to be installed
  
−
==== 28/01/2012 ====
+
==== 04/10/2015 ====
−
* Added iotop and cfv to be installed from apt repo.
+
* php.ini: Default memory max increased to 512MiB, upload max filesize to 16GiB and max simultaneous uploads to 50
−
* Added -m toggle for htpasswd because for some reason this is not default in Debian Lenny
 
−
* Added SSL support (self signed cert), just change http to https
 
−
* Added Webdav support, accessible from "/webdav-'''USERNAME'''/"  *** BETA: Permission issues, your mileage will vary
 
−
* Removed SSL key creation from install
 
−
* Update-step2 creates SSL key automatically utilizing the server hostname, removes old deprecated key
 
  
−
=== Changes 2011 ===
+
==== 02/10/2015 ====
 +
* addUser.php: refactoring and adding a chown to make sure everything is setup allright in regards of retrackers
  
−
==== 15/12/2011 ====
+
==== 01/10/2015 ====
−
* Added '''.irssi/config''' to user dirs, autojoining to #PulsedMedia on Freenode when irssi is launched
+
* update-step2: Another fix for retrackers related stuff
−
* Updated GUI IRC links, Chat tab and Adobe download links, with other small additional information and updates.
+
* userPermissions.php: Couple small fixes related to the above
−
* Removed Network Storage tab from GUI because Dropbox is blocking pageviews coming from our system for unknown reason.
 
−
* Added Proxy information on GUI
 
−
* Update-step2 sees if .irssi/config exists and if it does not copies it there
 
  
−
==== 30/11/2011 ====
+
==== 30/09/2015 ====
−
* Chosen rTorrent SCGI port range increased from 4k to 8k.
+
* Small update to rTorrent + libtorrent, faster tracker check interval default setting, and close idle connections faster
−
* "fractional blocks" for config. Resource calcs are based on number of blocks, and fractions were not allowed prior.
+
* install.sh: Refactoring to have all questions at the beginning
 +
* install.sh: Added many of the pkgs from update-step2 here, so we can later on remove them from update-step2
 +
* util/setupNetwork.php: commented out link speed determination - not required
 +
* install.sh: sysfs added net.ipv4.ip_forward = 1
 +
* setupNetwork.php: added enabling ipv4 ip forward
 +
* userPermissions.php: path fixes
 +
* addUser.php: Typo fix
 +
* (GUI) welcome.php: Updated with owncloud install link, and removed the need to visit setup-owncloud-finish.php
 +
* (GUI) Frames: Added owncloud tab if present
 +
* skel/www/setup-owncloud-finish.php: removed
 +
* update-step2: removed setup-owncloud-finish.php, and made owncloud setup file insertion conditional
 +
* (GUI) welcome.php: Organized the download links and fixed them :)
 +
* (GUI) welcome.php: Fixed quota warning var giving notices on error log
 +
* (GUI) stats.php: Changed getting IP to remote service
  
−
==== 29/11/2011 ====
+
==== 29/09/2015 ====
−
* During update also update user crontabs. Added to both update.php and step2, to be removed from step2 after this update has seen significant adoption.
+
* update-step2, addUser.php, userPermissions.php: Continuum of the earlier, apparently there's a slight logic error on ruTorrent preventing torrents directory from being created
−
* Restarting lighttpd during update, also done doubly for now
 
−
* Moved setup api key to regular update script. In several months it will potentially be removed from update all together as it's a installation step, and included to update older nodes.
 
−
* Scripts permissions update
 
−
* updateQuotas.php -> cron to update all user quotas, as discreet user from central cron to ensure it gets updated.
 
−
* Updated root  & user crontabs to reflect updateQuotas
 
  
−
==== 28/11/2011 ====
+
==== 28/09/2015 ====
−
* quotaFix now removes aquota*new files to avoid collision and thus failed update via quotacheck
+
* update-step2: Fixed permissions bug introduced by yesterday's changes
−
* userTorrents.php : Lists the count of torrents each user has, for quicker analysis usage stats
+
* addUser.php: Same fix as above
 +
* userPermissions.php: Fixing retroactively these. This is executed on server update.
  
−
==== 23/11/2011 ====
+
==== 27/09/2015 ====
−
* Permissions setup script added, which is utilized by recreateUser.
+
* checkInstances: Killing possible rTorrent processes fix.
−
* recreateUser.php fixes
+
* root cron: Change check Lighttpd for every 5 mins, delay beginning 30 secs
 +
* error-502: Wording change
 +
* addUser & update-step2: Adding retracker plugin config
  
−
==== 22/11/2011 ====
+
==== 04/09/2015 ====
−
* Created recreateUser.php which recreates home dir, preserving data+session if existing. ''ALPHA''
+
* createNginxConfig: Trim port number added
 +
* configureLighttpd: Increased potential port range
 +
* terminateUser: Remove lighttpd port config and nginx config
 +
* terminateUser: Some wording changes shown to user.
  
−
==== 19/11/2011 ====
+
==== 30/08/2015 ====
−
* Fixed saving API Key.
+
* Nginx: Custom 502 error page
 +
* util/createNginxConfig.php: Add template for proxy_params
 +
* Nginx: Increase proxy read timeout to 120s
 +
* skel/.lighttpd/php.ini: Increase max execution time to 90s and increase max memory to 256M per process! Increase Max file upload size from 2M to 32M, POST max size from 8MB to 32MB
 +
* update-step2: Fix owncloud setup files path.
 +
* update-step2: Add php.ini to the files to be updated
 +
* update-step2: Install packages flac, lame, lame-doc and mp3diags
  
−
==== 17/11/2011 ====
 
−
* Suspend and Unsuspend scripts fixed
 
−
* recreateLighttpd backup fixed
 
−
* Added access format of: '''servername/user-''username''/''', for example: ''localhost/user-johndoe/''. Thus supporting non FQDN setups
 
−
* addUser.php calls recreateLighttpd instead of just adding the hostname, this ensures lighttpd config is ''always'' up-to-date.
 
−
* Added to both update steps recreateLighttpd. It will remain on step2 for the next ~6months to ensure every system out there will have the update update.php
 
−
* Removed the default "testing" torrents
 
−
* Upload slots global, by default 6 times the non-global upload slots. It is ''"forcefully"'' inserted even if template does not it have for the moment. this forcefull insert will be removed in the future.
 
−
* checkInstances now also checks rtorrent configs, and if changed lists them on /root/changedConfigs for admin review
 
  
−
==== 25/8/2011 ====
+
==== 11/08/2015 ====
−
Streamlined installer for faster setups :) First release of API, just one command now. Some things moved to BETA status and considered feature complete. Many bug fixes, additional commenting etc. also added
+
* cron/diskIostat.php: Add disk quantity
 +
* cron/diskIostat.php: fix output camelCase
 +
* Added ownCloud installers - Install is now very simple and easy!
 +
* template.lighttpd: Added ownCloud directives for security
 +
* util/update-step2.php: Added update-rc.d lighttpd remove
 +
* util/update-step2.php: Added copying latest ownCloud setup files
 +
* cron/diskIostat.php: bug fix on device selection, on some edge cases the grouping was chosen incorrectly
  
−
* fixed userTransfer chowning lacking -R, thus UID was transferred for many files from the other server causing incorrect permissions
+
==== 10/08/2015 ====
−
* wrapped [[vnstat]] install into if !file_exists to see if [[vnstat]] is installed already.
+
* cron/diskIostat.php: Added cron job to gather current iostat info, 120 sec data
−
* added [[username]] 'srvadmin' to filter list on systemUsers
+
* cron.d/pmss: Run diskIostat every 15 mins
−
* removed from installer editing munin.conf -> if used can be done manually. Interim solution.
+
* util/userPermissions.php: Fixed xargs argument bug for delimiter
−
* put install [[vim]] as first thing on installer
+
* cron/diskIostat.php: Added logging to -history file as well for historic data, for potential later needs
−
* setting [[node]] [[hostname]] straight after installing vim
 
−
* moved setting up [[lighttpd]] as last thing
 
−
* scripts, skel, etc are installed earlier in installation
 
−
* [[SSL]] cert + [[quota]] as last items on setup
 
−
* Executing update-step2 on installer (so you don't need to execute updater after setup to get everything in)
 
−
* /scripts/util/setupRootCron.php -> creating root conf moved out of update.php
 
−
* setupSkelPermissions.php, same reason as root cron setup
 
−
* added setupRootCron and setupSkelPermissions to installer
 
−
* [[Idempotent]] lighttpd configurator feature completed. Backups the config first.
 
−
* Terminate user moved to beta status, as using lighttpd configurator now and restarting lighttpd.
 
−
* Added for terminate second argument, if 2nd argument is --confirm, straight accepts termination without asking for approval, for making automation easier.
 
−
* checkHttpd timer increased to 3 seconds before restarting lighttpd
 
−
* terminateUser timer increased to 5 seconds after trying to kill all user processes
 
−
* setupApiKey -> creates node key for API. Invoked upon setup.
 
−
* added authorization key
 
−
* removed /var/www/index.lighttpd.html
 
−
* added /var/www/index.php redir to pulsedmedia.com
 
−
* added beginnings of remote [[API]] for easier remote control management, by default in debug mode for now
 
  
 +
==== 02/06/2015 ====
 +
* userPermissions.php: Find all directories under user account and chmod them to 751 (default directory permissions: rwxr-x--x).
  
−
==== 8/6/2011 ====
+
==== 06/04/2015 ====
−
Bug fixes.
+
* userPermissions.php: Chmod .ssh if the directory exists
  
−
terminateUser.php:
+
==== 05/04/2015 ====
−
* Was not actually completing shell userdeletion
 
−
* Added killing user processes (forgotten)
 
−
* added sleeping 2 seconds to allow time for rTorrent/other processes by user to die
 
−
* Added 2nd attempt to userdel after data removal
 
−
* Fixed some typos in texts
 
−
* Added check for user actually exists
 
−
'''NOTE: Still Alpha, non feature complete and not recommended for production use.'''
 
  
−
* checkHttpd.php added sleep between lighttpd kill and restart, to avoid PID file mismatch scenario
+
* userPermissions.php: Chowning as well to make sure right user & group owns the directory.
 +
* updates-step2.php: Added ZNC and ZNC-EXTRA packages
  
−
update-step2.php:
+
==== 02/04/2015 ====
−
* First time being used
 
−
* update.php moved cron & ftp config stuff AFTER step2
 
−
* Install vnstat + vim (vim not by default on Debian 6)
 
−
* Setup interface for vnstat
 
−
* Configure vnstat for Debian 6
 
−
* Make vnstat cron jobs for Debian 5: twice a minute update (thus no support for 1GigE interface)
 
−
* Debian 5 add cron jobs to root.cron.additional config, create if needed
 
−
 
 
−
==== 31/5/2011 ====
 
  
−
* GUI welcome page checking if proper quota information was passed has been added.
+
* configureLighttpd.php: Typo fix
−
* GUI welcome page malformed serialized data fix
 
−
* GUI welcome page quota percentage missing % char
 
−
* GUI welcome page quota text styles: White and location
 
  
−
==== 30/5/2011 ====
+
==== 01/04/2015 ====
 +
Some bug fixing to yesterday's release
  
−
* GUI update script quota checking added (doesn't try to update when user is over quota burst limit, no more blanking GUI when update happens!)
+
* userPermissions.php: chmod 771 couple temp dirs for each user's lighttpd
−
* Users default quota information update rate has been increased to 10mins
+
* update-step2.php: apt-get remove go git mercurial added since these packages are not needed
−
* HDD Quota plugin commenting added
 
−
* HDD Quota plugin parser bug with bursted amounts fixed
 
−
* HDD Quota now shows USED/QUOTA instead of FREE/QUOTA. Makes more sense.
 
−
* HDD Quota "title" fixed (showing quota info) to be whole area of the meter
 
−
* Welcome page quota meter added :) Basic graph, larger and constantly visible text for quota info :)
 
−
* Basic [[Idempotency|idempotent]] lighttpd configurator (alpha, not suitable for production, TESTING only)
 
  
−
==== 23/5/2011 ====
 
  
−
* terminateUser.php http password deletion fix, text fixes, executability.
+
==== 31/03/2015 ====
−
* checkHttpd.php output lighttpd output, killall -9 lighttpd to ensure shutdown, http fetch mute removed so error gets logged
+
This release is all about converting to per user lighttpd processes and several bug fixes, and hence enhancing service quality and security while opening the door for future new feature developments. This also adds capability for public web hosting from the seedbox.
−
* update.php: removed apt updates because soft.sh already does this
 
−
* crons: increased check http to 4 minute intervals (avoid http timeout period issues)
 
  
−
==== 18/5/2011 ====
+
* install.sh: remove packages golang git mercurial
−
Temporary fix to bypass userDb until fully utilized. You may use this update or use /scripts/util/populateUserDb.php to fix the issue.
+
* update-step2: remove packages golang git mercurial
 +
* install.sh: also put into fstab vfsv1 instead of vfsv0
 +
* update-step2: remove migration to vfsv1 since this is prone to breaking quota data. need to look for better migration method.
 +
* etc/seedbox/config/template.lighttpd added and created
 +
* etc/skel/lighttpd directory added
 +
* etc/seedbox/config/template.nginx-user added and created
 +
* etc/seedbox/config/template.nginx added and created
 +
* util/configureLighttpd.php: Created configurator for per user lighttpd configs
 +
* etc/skel/lighttpd/compress directory added
 +
* etc/skel/lighttpd/upload directory added
 +
* template.lighttpd: both password protected (normal GUI) and public web directories included in the config where user can run whatever they please.
 +
* changePw.php: changes now the per user htpasswd file (username/.lighttpd/.htpasswd)
 +
* addUser.php: use changePw.php script instead of set in the file
 +
* addUser.php: if password rand has been passed, use a random password.
 +
* addUser.php: remove chmod rules
 +
* addUser.php: if runtime/trafficLimits dir does not exist, create it
 +
* addUser.php: use configureLighttpd.php script instead of setting it up hardcoded within the script
 +
* addUser.php: change expire date to 2100-01-01
 +
* addUser.php: Do not add user to www-data group
 +
* util/createNginxConfig.php: Create per user proxy config files (idempotent) and set default nginx config from template
 +
* createNginxConfig: Create SSL config.
 +
* template.nginx: Include SSL config
 +
* scripts/startLighttpd added and created
 +
* scripts/cron/checkLighttpdInstances.php added and created
 +
* update-step2: Added all the lighttpd configuring, installing packages, managing processes etc. there. This should be the bulk of conversion
 +
* addUser.php: create nginx config, and restart nginx
 +
* config/root.cron: Comment out checkHttpd for now and add checkLighttpdInstances in there
 +
* userConfig.php: Do not launch traffic limit script if none has been passed
 +
* lib/rtorrentConfig.php: If no logger, do not echo the contents (rel: addUser.php needlessly echoing rtorrent config)
 +
* changePw.php: if password defined it's not set to correct variable and thus cannot set password
 +
* changePw.php: Also check for existence of per user htpasswd file!
 +
* util/checkUserHtpasswd.php: Created to move the user's password from global to per user.
 +
* update-step2: run checkUserHtpasswd.php and restart all lighttpd processes
 +
* update-step2: lots of permissions updates
 +
* update-step2: remove some deprecated, commented out code
 +
* util/userPermissions.php: created this to set all the folder and file permissions for the user directory. Now they are in one place!
 +
* addUser.php: use userPermissions.php
 +
* update-step2: use userPermissions.php
 +
* template.lighttpd: Use per user php.ini and change fcgi settings a bit
 +
* skel/.lighttpd/php.ini added
 +
* configureLighttpd.php: Add .lighttpd dir if it doesn't exist yet, separately check for php.ini as well.
 +
* update-step2: Do not remove autotools plugin if it is present
 +
* update-step2: add package ntp
 +
* checkUserHtpasswd.php: Chown the newly created .htpasswd file
 +
* userPermissions.php: set htpasswd file perms as  well
 +
* update-step2: move earlier in the process to make the move to nginx etc. for working conversions. make sure lighttpd processes are stopped.
 +
* update.php: remove recreateLighttpdConfig & restart, replace with nginx
 +
* release packager: include .lighttpd directory into the skel package.
 +
* userPermissions.php: some chowns and extra rules required for smooth conversion
 +
* terminateUser.php: replace lighttpd related commands with nginx replacements
 +
* terminateUser.php: remove deleting htpasswd. Add deleting home dir in case userdel does not remove it
 +
* recreateUser.php: Changed to fit the new model.
 +
* configureLighttpd.php: Also check for public html directory
 +
* update-step2: Do the configs twice to be certain, as another update step might have affected.
  
−
* Changed listUsers.php to always use systemUsers.php, as fixing userDb saving introduced a bug where ie. checkInstances was not working fully (using userDb, which lacks most users in most servers)
+
==== 07/03/2015 ====
 +
* quotaFix.php: added -n flag for quotacheck to automatically answer questions
 +
* install.sh: Update quota vfsv0 to vfsv1 to support large quotas
 +
* update-step2: Update quota vfsv0 to vfsv1 to support large quotas, and commit change + execute quotaFix
 +
* update-step2: Check if user is suspended, if so do not make updates
 +
* update-step2: Reinstalling unpack plugin for ruTorrent which was earlier removed for abuse causing extremely high loads on a server
  
−
==== 16/5/2011 ====
+
==== 27/02/2015 ====
 +
* update-step2: moved the location of nzbdrone repos to the normal repo list, cannot move sonarr because https transport needs to be first installed.
 +
* update-step2: Refactored stopping & disabling services not needed for "global" user
 +
* update-step2: added apt-get -f install -y to fix potentially broken or missing dependencies
 +
* update-step2: Forgot to add -y toggle for sabnzbdplus package install requiring manual intervention
  
−
Few bug fixes and minor features introduced.
+
==== 26/02/2015 ====
−
* Update script order of items.
+
* install.sh: Added locate apt package, since some minimal installs are missing this
−
* Update script apt-get options for additional package installation lacked auto confirm.
+
* update-step2: Added locate apt package
−
* Update script support for ''"step 2"'', /scripts/util/update-step2.php is included if present, thus supporting dynamic updates
+
* install.sh: Added python-pycurl, python-crypto, python-central packages
−
* Update script execute binary line added, so it can actually be executed simply by ./update.php
+
* update-step2: Install pyLoad and dependencies python-pycurl, python-crypto, python-central, golang, git, mercurial
−
* /scripts permission fix, only root should have anykind of access (no execute or read for others)
+
* install.sh & update-step2: Install packages gcc g++ gettext python-cheetah curl fuse glib-networking libglib2.0-dev libfuse-dev apt-transport-https
−
* Support for root additional items via /etc/seedbox/config/root.cron.additional
+
* update-step2: Compile & install Megatools from source
−
* Add user script minor cleanup, additional commenting.
+
* update-step2: Re-ordered package installs as some installation & compilations were attempted BEFORE installing the dependiences.
−
* Suspend script comments
+
* update-step2: get, compile and install Mono 3.8.0
−
* simple terminate user script, ALPHA -> not feature complete yet.
+
* update-step2: Install Sonarr / Nzbdrone. To start it execute '''mono /opt/NzbDrone/NzbDrone.exe''' and then browse to '''http://servername.pulsedmedia.com:8989''' - remember to change port and set security + authentication credentials!
−
'''Updating from 27/03/2011 release or earlier'''
+
* update-step2: To make sure they are not running for non-specific seedbox user, stop btsync and pyload, and update-rc.d disable them
−
cd ~; bash soft.sh; php /scripts/update.php
+
* update-step2: Install sabnzbdplus
  
−
'''Updating from 15/5/2011 release'''
+
==== 02/02/2015 ====
−
php /scripts/update.php
+
* update-step2: ntpdate for some reason was not installed by default - included this on update
 +
* update-step2: now vnstat config is checked on every update
 +
* update-step2: vnstatConfig also sets maxbandwidth higher
 +
* apt-get clean & update for some reason missing from update.php - added, and also temporarily for step2
  
 +
==== 31/01/2015 ====
 +
* Security update on lighttpd config.
  
−
==== 15/5/2011 ====
+
==== 30/01/2015 ====
 +
* btsync sample config now in skel as .btsync.conf-sample
 +
* btsync binary added to /usr/bin/btsync now.
  
−
Couple important new features introduced, several bug fixes.
+
==== 21/01/2015 ====
−
* Added  [[lighttpd]] status checking and auto-restart (very rudimentary)
+
* rutorrent/share/settings/rss folder was missing, created to skel and added to update for creating it
−
* Fixed typo in /scripts/util/userConfig.php
 
−
* Added /etc/seedbox/config/root.cron
 
−
* /scripts/util/quotaFix.php now echoes results (also turns [[quota]] off, no need to turn it off first)
 
−
* [[proftpd]] config update (remote) with MaxClientsPerUser, MaxClientsPerHost, run /scripts/util/ftpConfig.php
 
−
* users class now supresses error if runtime db is not found
 
−
* users class now saves the user db upon save
 
−
* users class deconstructor added to save users db upon exit
 
−
* modules support code begun (version for highly dynamic with multiple hooks etc.)
 
−
* Changed update.php to a very basic one, as the remote capabilities are not being employed currently
 
−
* Rudimentary module ''/etc/seedbox/module/basic/addUser.php'' for ultra simplistic user addition module (ie. custom configs, limitations).
 
−
* Basic suspend & unsuspend capabilities (untested ALPHA)
 
−
* skel permissions fixed
 
−
'''updating'''
 
−
bash soft.sh; php /scripts/update.php
 
  
 +
==== 19/01/2015 ====
 +
* trafficStats: Added logging also date & time. For some reason this was not logged.
 +
* trafficLog: removed serverApi calls as cnc is not utilized anymore
 +
* skel/www/rutorrent/plugins/check_port removed as well. was prior removed only on update. this plugin doesn't function anymore as expected.
 +
* some debug logging added for trafficStats cron
  
  
−
==== 27/03/2011 ====
 
  
−
This is primarily a bug fix and code quality release.
+
=== Changes 2011-2014 ===
−
* Set shell to /bin/bash upon user creation
+
Can be found at [[PM Software Stack Changelog 2011-2014]]
−
* Set shell to /bin/bash when using userConfig
 
−
* quotaFix.php now turns quota off before attempting quota check
 
−
* quotaFix.php now displays quota info before and after check
 
−
* Lots of comments added to all code files touched
 
−
* userConfig.php $filesLimitPerGb was undefined - hardcoded 500
 
−
* update.php minor changes
 
−
* default version now set to 1, in preparation to take it into production
 
−
* startRtorrent: codestyle fixes, Made error messages more clear, named rTorrent screen session to "rtorrent"
 

Latest revision as of 15:05, 4 February 2023

PMSS, short of Pulsed Media Software Stack is a compilation of scripts to manage single server side seedbox configuration, turning instantly and conveniently an regular Debian server into a seedbox. It is free to use for anyone, see Installing PM Software Stack.

PMSS supports rTorrent+ruTorrent, Deluge, qBittorrent, rclone etc.

Contents

General features

Primary design goals are reliability and ease of server management.

Using PMSS

See Category:PM Software Stack Guides

Debian 8 notes

Debian 8 support is considered now EOL -- No new patches will be made for Debian 8.

Debian 10 notes

Debian 10 considered stable. In production.

Changelog

Changes 2023

NOW ON GITHUB

See https://github.com/MagnaCapax/PMSS work and changelog continues on there.

01/2023

  • /etc/skel/.bashrc: Add ~/bin to path
  • /scripts/util/userConfig.php: Systemd enable linger, install docker rootless + docker-compose for the end user
  • /scripts/lib/apps/docker.php: Added, install docker rootless related stuff

Changes 2022

15/12/2022

  • /scripts/lib/apps/packages.php: add package uidmap

15/12/2022

  • /scripts/lib/apps/packages.php: Changed curl to backports, rtorrent recently started crashing again frequently for a lot of people, so update to curl fixed this.

15/12/2022

  • scripts/util/userTransfer.php: Exclusion list extended a little bit more

7/12/2022

  • scripts/util/userTransfer.php: Exclusion list extended to various runtime and configs which were mistakenly being migrated over (the main issue with "catchall" type of migration)

2/12/2022

  • scripts/util/userTransfer.php: Second step migration added ~/.lighttpd/custom.d and ~/www/public directories

23/11/2022

  • scripts/util/update-step2.php: Remove _user_/www/phpXplorer symlink, very very old compatibility thing
  • skel/www/phpXplorer removed

15/11/2022

  • scripts/util/userTransfer.php: Moved .local to last step, some refactoring+commenting, and multiple attempts on 2nd step too. Increased timers in case source server issues

15/11/2022

  • scripts/util/userTransfer.php: No need to insert username as double anymore, conditionally assume it is the same
  • scripts/util/userTransfer.php: Completely new transfer, attempt to transfer everything. First beta let's hope stuff doesn't break too much
  • scripts/util/userTransfer.php: Increase delay between re-attempts, and increase count a bit too
  • etc/seedbox/config/template.user-slices-pmss.conf: Tasks max lowered to 250 processes to limit fork bombing done by some users

19/10/2022

  • scripts/util/update-step2.php: Set systemctl user slice basic defaults file
  • scripts/util/userConfig.php: Create user slice configs, CPU, IO Weights and memory soft & hard limits. New additional config options for cpu + ioweight, memory soft limit is advertised memory limit and memory hard limit is twice that.
  • etc/seedbox/config/template.user-slice.conf: Added template for the user slice
  • etc/seedbox/config/template.user-slices-pmss.conf: Default user slice config to be added to systems
  • etc/skel/www: php files updated to latest versions (the new theming is earlier user contribution)
  • etc/skel/rutorrent/plugins/theme/themes: Added themes from https://github.com/artyuum/3rd-party-ruTorrent-Themes and made MaterialDesign the default (user contrib)
  • scripts/util/update-step2.php: Update themes default, and add the new ones if missing
  • scripts/lib/packages.php: Update to MediaInfo repos and install newest available mediainfo package (user contrib)
  • scripts/lib/packages.php: Buster backport kernel update to 5.10 series

18/06/2022

  • scripts/lib/apps/deluge.php: Add debian packages python & python3 setproctitle for deluge installation.

04/05/2022

  • scripts/util/update-step2.php: Install latest rc.local on each update

30/04/2022

  • scripts/terminateUser.php: Additional username checks

13/03/2022

  • scripts/util/userPermissions.php: Changes on .config, .tmp -- as expected there's been regressions. some other small changes too
  • skel/.bashrc: Default to color prompt, ls colors and set locales
  • scripts/util/update-step2.php: Set locales. Some installers don't setup as UTF-8 by default. Update .bashrc automatically

12/03/2022

  • scripts/util/userPermissions.php: Fixed directory 751 to 750 permissions, and fixed home directory permissions order being incorrect. **Untested** but due to public disclosure before tested fix, we have to push this out.
  • install.sh: Add mount proc with hidepid **this will most likely break some other things now -- untested, but same reason as above**
  • scripts/util/userPermissions.php: added ~/.tmp + .config to the list of chmods as well -- these are relatively new directories by new additional software added as of late, and those defaults might not be the best
  • addUser.php: Leave userPermissions run into the background, do not lock the process.

06/03/2022

  • scripts/cron/checkDelugeInstances.php: Refactoring and 2 typoes on execution commands

21/02/2022

  • GUI: Custom frames support! ~/.customFrames file with format, one per line: appname|tooltip|label|url -- full documentation on ~/www/index.php on the format. User contribution.
  • etc/seedbox/config/template.qbittorrent.conf: Change for random external port
  • scripts/util/userConfig.php: Only config qBittorrent when config does not exist -- may cause an issue when reconfiguration is needed tho
  • GUI: info page stats: RAM usage reformatted for nicer look

20/02/2022

  • GUI: Layout changes. User contribution.
  • GUI: Fixed height issues and removed news tab
  • GUI: Welcome page now has announcements rss feed

19/02/2022

  • etc/seedbox/config/root.cron: Added bash, deluged, qbittorrent-nox, xmrig CPU & I/O scheduling rules
  • skel/.bashrc: added arrinfo alias for listing URLs if using the community script to install everything. User contribution.
  • www/lighttpdRestart.php: Change lighttpd restart to SIGUSR1 -- faster and cleaner. User contribution

15/02/2022

These are user submitted

  • util/configureLighttpd.conf: Create .lighttpd/custom.d
  • skel/.lighttpd/custom.d: directory added
  • util/update-step2.php: chown some log files, and bins for admin access only
  • seedbox/config/template.lighttpd: Add include .lighttpd/custom.d/*.conf for user customization purposes

11/02/2022

  • Welcome page: Changed rTorrent restart button to middle and added lighttpd restart button
  • seedbox/config/template.nginx-user: Add 502 error page directive
  • var/www/error-502.html small text update

Changes 2021

20/12/2021

  • scripts/util/userPermissions.php: Reverted chown command back

16/12/2021

  • scripts/util/userPermissions.php: Remove -R from chmod 640 user home dir -- this was potentially messing up some emby & jellyfin installations -- also remove an chown -R directive (redundant).
  • config/template.rutorrent.config: Increase timeouts and change log file

29/11/2021

Added qBittorent and bug fixing / performance increases for lighttpd

  • www/filemanager.php: Slight optimization on the download file routines
  • etc/seedbox/config/template.lighttpd: Add stream configs, and define fastcgi min_procs, increase max_procs
  • lib/apps/packages.php: Remove avahi-daemon, install qbittorrent-nox
  • etc/seedbox/config/template.lighttpd: Added qbittorrent reverse proxy settings
  • util/configureLighttpd.php: Added qBittorrent config
  • util/userConfig.php: Added qBittorrent config
  • cron/checkQbittorrentInstances.php added
  • etc/seedbox/config/root.cron: Add check qBittorrent instances

19/11/2021

  • scripts/cron/diskIostat.php: fix deb10 collected fields

12/10/2021

Log file is at ~/.rcloneLog Issue for sometimes getting 500 internal server error was because in some cases using ~/ did not function, where as at other times it works perfectly. Changed all of them to full paths using 'whoami' Did almost the same for deluge

  • scripts/cron/checkRcloneInstances.php: Add logging, full path to htpasswd aswell
  • www/rclone.php: Add logging (comes on update, not on distribution package), full path to htpasswd aswell
  • www/deluge.php: Add logging, also comes via an central update not distribution package
  • scripts/cron/checkDelugeInstances.php: Add logging

28/09/2021

  • scripts/cron/trafficLog.php: New grep rules had typo, which caused no effect
  • scripts/cron/trafficLog.php: Refactoring a little bit
  • scripts/cron/trafficLog.php: Additional checks and debugging
  • /scripts/lib/networkInfo.php: Default to 1000Mbps if couldn't determine otherwise
  • scripts/cron/trafficLog.php: Log if user 5min average exceeds 25% link maximum (so we can more easily find when the errors occur!)
  • scripts/cron/trafficLog.php: Additional grep rules for data collection, just in case. Identified that occasionally iptables throws an error, and that make throw us a complete curve ball
  • scripts/cron/trafficLog.php: Only try to reset OUTPUT chain stats. Identified miscounting is related on iptables failing to reset counters with invalid argumen on INPUT chain
  • scripts/util/makeMonitoringRules.php: Change initial mark
  • etc/seedbox/config/template.fireqos: Changed users to balanced priority
  • Syncthing distribution version changed to latest
  • scripts/lib/apps/syncthing.php: Check which version and update
  • scripts/lib/apps/rclone.php: Update to version v1.56.1
  • scripts/lib/apps/syncthing.php: Variable name type fixed
  • scripts/lib/apps/filebot.php: Updated to verion 4.9.4
  • scripts/util/userConfig.php: Put randomized rclone port (user can change this too)
  • scripts/util/configureLighttpd.php: Stop making so many backups
  • GUI tabs include rclone if enabled
  • GUI welcome page has rclone control buttons
  • GUI added rclone.php
  • etc/seedbox/config/template.lighttpd: Added rclone section
  • etc/seedbox/config/template.lighttpd: Increased per connection speeds
  • etc/seedbox/config/template.lighttpd: Disabled fcgi debug
  • scripts/util/configureLighttpd.php: Add rclone section
  • script/cron/checkRcloneInstances.php: Added
  • etc/seedbox/config/root.cron: Added check rclone instances every 2 minutes

23/09/2021

  • scripts/cron/trafficLog.php: trim and format logged output values. On rare occasion there was extra line changes.
  • scripts/cron/trafficLog.php: Omit QOS MARK lines, this might've been occasionally the reason for miscalculation tho we cannot reproduce the issue
  • etc/seedbox/config/root.cron: Change traffic limit checks to once per hour, and traffic stats generation twice per hour
  • scripts/cron/trafficLimits.php: Silence error if trying to disable limit which is not enabled to begin with
  • scripts/cron/checkLighttpdInstances.php: Now also check with curl that we get 401 Unauthorized response.
  • scripts/cron/checkInstances.php: Check for rtorrent socket file now as well, if instance still found kill it.
  • scripts/util/update-step2.php: Add for deb10 apt update to get minor version to update, and remove apt-get dist-upgrade command

22/08/2021

  • scripts/lib/apps/packages.php: Add python3-venv
  • scripts/util/setupLetsEncrypt.php: certbot changed again (EFF sure likes to break old users, don't they?), change over to venv/pip method. Since it's pip etc. let's hope it works for more than a week.
  • scripts/lib/apps/packages.php: Remove openvpn package
  • scripts/lib/apps/openvpn.php: disabled until support is fixed

17/04/2021

  • scripts/lib/apps/packages.php: Fixed bug where some packages were not installed for Debian 10

10/04/2021

  • scripts/util/setupLetsEncrypt.php: Changed around how it functions as certbot changed and the certbot auto script does not function anymore. Now certs should get installed again normally on Debian 10 systems. Debian 8 not supported at this point (untested)

24/02/2021

  • scripts/cron/checkInstances.php: Comment out the pid file checking, not sure if this is necessary to date but may cause multiple executors issue
  • apps/packages.php: Add zsh and atop
  • scripts/cron/userTrackerCleaner.php: Check if there is existing instances, if there is do not launch
  • scripts/cron/checkInstances.php: Check against executor script instead of rtorrent instance itself. Fixes a issue where there might be hundreds of executor scripts running

Changes 2020

10/12/2020

  • lib/apps/deluge.php: Deb8 backwards compatibility fix

26/11/2020

  • lib/apps/deluge.php: Some backwards compatibility fix with Deb8 repo package symlinks
  • lib/apps/packages.php: Deb10 compatibility fixes, add php-mbstring package
  • lib/apps/deluge.php: path typo fix for deb10 systems

23/11/2020

  • lib/apps/deluge.php: Fixed compilation bug
  • util/setupUserHomePermissions.php: removed some of the rules

12/11/2020

  • util/userConfig.php: Various fixes for deluge configuration + update nginx config and restart nginx
  • cron/userTrackerCleaner.php: Lower the number of checks even lower, just 2 users and max 20 torrents at a time
  • etc/seedbox/config/template.deluge.web.conf: Typo fix and base url fix

10/11/2020

  • util/userConfig.php: Typo choosing deluge port fixed

09/11/2020

  • lib/apps/deluge.php: Use repo version for non debian 10 installs
  • util/userConfig.php: Fix cache allocation, it is 16KiB blocks not MiBs

08/11/2020

Tracker Cleaner enabled ++ Deluge Install

  • cron/userTrackerCleaner.php: First filter list -- AND THIS IS NOW ENABLED.
  • lib/apps/rtorrent.php: If not Debian 10, try 0.9.6. This is part of debugging to see if 0.9.6 could be used again
  • lib/apps/deluge.php: Install deluge, in the most basic form no fail checks etc. hopefully works on all deb8/deb10 systems despite pythoness issues. There is some redundant commands here, but we'll fix them later
  • etc/seedbox/config/template.deluge*: Added base templates for Deluge
  • util/userConfig.php: Added base deluge configuration
  • etc/seedbox/config/template.nginx-user: Add deluge proxy settings for /deluge-USERNAME/
  • cron/checkLighttpdInstances.php: Logging said start rtorrent, fixed
  • cron/checkDelugeInstances.php: Start deluge instances if enabled via /home/USERNAME/.delugeEnable file existing
  • etc/seedbox/config/root.cron: Add checkDelugeInstances at 2 minute interval
  • +GUI update logging
  • +GUI deluge buttons + tab

06/11/2020

  • seedbox/config/root.cron: check lighttpd instances once per minute
  • cron/checkLighttpdInstances.php: Check also php-cgi, and restart if php-cgi not present
  • seedbox/config/template.lighttpd: Back to more sensible fcgi parameters
  • var/www/error-502.html: Fixed the wait time numbers
  • cron/userTrackerCleaner.php: Cleaning of udp trackers from public torrents which block the whole rTorrent instance. This can be disable by creating .trackerCleanerDisable on user home directory
  • scripts/lib/devristo directory: library from https://github.com/Devristo/torrent/ to manage torrents
  • skel/.rtorrent.rc.custom: By default again create the .logs directory and log tracker notices, storage and socket critical errors
  • cron/checkLighttpdInstances.php: remove 2 sec delay between start events for each user
  • skel/.rtorrent.rc.custom: Turns out the logging levels are buggered too, only debug level shows any tracker action *sigh*
  • util/update-step2.php: Check for original empty rtorrent.rc.custom; and instead of deleting the old one, update with new one
  • seedbox/config/root.cron: added tracker cleaner, but that script doesn't yet do anything, we need to build the list first

02/11/2020

  • util/update-step2.php: Remove the way too verbose logging added on .rtorrent.rc.custom
  • skel/.rtorrent.rc.custom: Comment out the logging stuff

01/11/2020

Some major changes: New rTorrent (latest git, udns branch) with some changes in how it is compiled. New traffic control mechanism via QoS since rTorrent limiting did not since update to 0.9.8

  • lib/apps/rtorrent.php: Fix typo
  • lib/apps/rtorrent.php: More typoes fixed
  • lib/apps/rtorrent.php: rtorrent version checked here and target version variable to make updates easier
  • util/update-step2.php: remove checking rTorrent version as only the installer requires this
  • lib/apps/rtorrent.php: Add check if xmlrpc-c already installed, if not do not install again
  • util/setupNetwork.php: Clear all rules at start
  • util/makeMonitoringRules.php: Add connmark for users, for which no local network was matched for
  • etc/seedbox/config/network: Added basic network config to remove autodetect issues in future, default eth0 and 1Gbit speeds defined
  • util/update-step2.php: Notification to edit this file
  • util/update-step2.php: Moved various end of update stuff here, as /scripts/update.php should be as sparse as possible
  • update.php: Moved some stuff to update-step2
  • util/makeMonitoringRules.php: Added packet marking
  • util/setupNetwork.php: Added qos rule creation, and starts qos
  • cron/trafficLimits.php: Old timey rtorrent traffic limit removed, only puts limit info on user account now
  • util/setupNetwork.php: Does traffic limiting now on qos level
  • cron/trafficLog.php: typo fixed
  • lib/apps/rclone.php: well, path fix needed fixing :/
  • lib/networkInfo.php: changes to use the network config file

30/10/2020

  • etc/seedbox/config/template.lighttpd: Change fastcgi settings to 1000 max requests.
  • etc/seedbox/config/template.lighttpd: Enable fastcgi debugging in hopes to catching why fastcgi crashes on some servers
  • scripts/lib/apps/packages.php: Added php7.3-xml php7-3.zip for debian 10 packages install
  • rTorrent / ruTorrent: Back to 0.9.8; udns branch did not solve any issues

06/10/2020

  • skel/www/filemanager.php: Enforce https

05/10/2020

  • etc/seedbox/config/template.lighttpd: Increase performance + stability, both PHP and throughput via altered settings. php-cgi with new rutorrent is crashing, increasing process count while decreasing max requests per children crashes will become less frequent likely.

13/09/2020

  • cron/diskIostat.php: Debian 10 iostat mapping
  • www/rutorrent/plugins/hddquota/action.php: Update for debian 7, regexp modifiers changed
  • NEW FILEMANAGER! Using installed as "filemanager.php" and GUI updates to that if it's been installed.

05/09/2020

  • util/userTransfer.php: Change the rsync spawn line authentication info, bug fix for Debian 10

27/08/2020

This makes Debian 10 production ready with 2 caveats: No OpenVPN and Mega tools.


  • etc/seedbox/config/template.lighttpd: changed custom file from include_shell to just include (regular config file instead of shell script). New version of lighttpd fails to start unless a script is present, but empty file is ok
  • etc/seedbox/config/template.lighttpd: changed include_shell for create-mime to include wildcard as Debian 10 lighttpd has it renamed AND now fails to start if any file is missing from includes. this should work with debian 8 and debian 10

05/08/2020

  • lib/apps/radarr.php: Bug in installer fixed, tried to get the installer package from wrong path. Updates to the latest version now on each server update as well.
  • util/userTransfer.php: Mistake in the rsync commandline fixed

20/07/2020

  • .rtorrent.rc.custom: Added logging by default due to some regression with rtorrent lately -- users can freely disable this.
  • util/userTransfer.php: Added migration of rtorrent.rc.custom as well.
  • skel/www/rutorrent: Update to ruTorrent V3.10 -- reintroduce check_port plugin as well as it seems to have received patches now. Include all standard plugins except diskspace and _cloudflare
  • lib/update.php: Some updates on rutorrent config.php configs
  • util/update-step2.php: Re-add updating rutorrent during this process
  • util/update-step2.php: Install Let's Encrypt cert during update
  • lib/update.php: Comment out rutorrent filemanager plugin stuff for now - retest at later date for addition

16/07/2020

Ouch! As part of DDoS mitigation we started filtering other subnets as well, and we had blocked the network we utilize for OpenVPN connectivity. Removed that specific filtering as temporary measure.

  • util/setupNetwork.php: Removed filtering 10.0.0.0/8

15/07/2020

Core functionality for Debian 10 should work now, some applications need to still be checked but all core stuff should function as expected.

  • lib/apps/packages.php: Install expect
  • util/update-step2.php: move rTorrent compilation to the lib/apps/rtorrent.php file and use that
  • lib/apps/rtorrent.php: Add this file + Install & compile xmlrpc-c r3116 (bumping version)
  • util/update-step2.php: Refactor ipset, iprange, firehol, filebot to lib/apps

11/06/2020

Change rtorrent to latest git and libtorrent to the udns branch

  • Install.sh: Comment out compiling libtorrent + rtorrent
  • lib/update-step2.php: bump to libtorrent udns branch
  • lib/update-step2.php: Delete check_port rutorrent plugin
  • rutorrent: remove check_port plugin from default plugins. This plugin causes a lot of confusion as it is broken most of the time.
  • rclone version bumped to 1.52.1

27/03/2020

Basic debian 10 buster support. Experimental still, requires a lot of testing.

  • listUsers.php -> fix interpreter path
  • addUser.php -> fix interpreter path
  • changePw.php -> fix interpreter path
  • filecopy.php -> fix interpreter path
  • suspend.php -> fix interpreter path
  • unsuspend.php -> fix interpreter path
  • recreateUser.php -> fix interpreter path
  • systemUsers.php -> fix interpreter path
  • cron/checkHttpd.php -> fix interpreter path
  • util/populateUserDb.php -> fix interpreter path
  • util/setupUserHomePermissions.php -> fix interpreter path
  • install.sh: Debian 10 support and some cleanup of old deprecated stuff
  • util/checkUserHtpasswd.php: quiet error if lighttpd htpasswd file not found & commenting
  • lib/apps/packages.php: Debian10 support, some refactoring
  • lib/apps/mono.php: install package mono-complete, move deb8 mono install code from sonarr.php here
  • lib/apps/sonarr.php: Remove mono installation thing
  • lib/apps/radarr.php: Rudimentary radarr installer, but no config or testing yet
  • util/update-step2.php: Some refactoring, Debian 10 repos, commenting old deprecated code etc.

23/03/2020

21/02/2020

  • util/update-step2.php: Install tree, ncdu and weechat

20/02/2020

  • util/update-step2.php: Install tmux

07/02/2020

  • .rtorrentExecute.php: Add failsafe checks for rtorrent or executor already running, if more than 1x rtorrent running kill all rtorrent instances.
  • lib/rtorrentConfig.php: Remove rtorrent.rc custom option from here, it can be done via config
  • skel/.rtorrent.rc.custom: Added
  • rtorrent.rc template changed to try loading this file
  • rtorrent.rc template "modernized", and things moved around etc.
  • minimum peers lowered to just a few
  • template.nginx-conf: keepalive timeout 65s -> 20s, client max body size 4096M -> 8192M, TLSv1 removed, SSLv3 added
  • template.nginx-site-default: HTTPS/SSL Section separate to template.nginx-site-default-default-ssl, replace with placeholder which will cause nginx not to launch unless configured by configurator
  • template.nginx-site-default-ssl: added with original config, except protocols changed and ciphers matching let's encrypt
  • util/createNginxConfig: make building https/ssl config around either self signed cert, or let's encrypt
  • util/setupLetsEncrypt.php: A quick let's encrypt cert install script. Pass e-mail address and all should function

06/02/2020

  • lib/rtorrentConfig.php: Add support for .rtorrent.rc.custom - this is included on top of the config file, so resource settings should get overwritten and remain same

02/02/2020

Bonding interface support

  • lib/networkInfo.php: Network information fetching snippet
  • util/setupNetwork.php: Refactor link determination stuff to networkInfo.php. Fix static definitions for eth0
  • lib/apps/vnstat.php: Use networkInfo output for link + speeds
  • cron/trafficLog.php: Use linkspeed to determine maximum usage.

Changes 2019

08/12/2019

More rTorrent regression fixes for 0.9.8 -> simple renamings of old config options etc.

  • lib/rTorrentConfig.php: Use new template with these fixed, and for memory instead of bytes define as megabytes

06/12/2019

rTorrent regressions fix for 0.9.8, described at: https://github.com/Novik/ruTorrent/commit/ec8d8f1887af57793a671258072b59193a5d8d6c

  • change skel according to above patch
  • util/update-step2.php: update above user files

05/12/2019

  • Remove use_udp_trackers and extranous use_udp_set from .rtorrent.rc template, and bump date.
  • lib/rtorrentConfig.php: Use the new version
  • util/update-step2.php: Bump rtorrent version to 0.9.8 libtorrent to 0.13.8
  • util/update-step2.php: Remove use_udp_set = yes from local template AND from users
  • install.sh: Bump rtorrent version to 0.9.7 libtorrent to 0.13.8
  • util/update-step2.php: Fix updating repos and add dist-upgrade -y
  • etc/seedbox/config/template.openvpn.server.config: change to use aes-256-cbc cipher
  • lib/apps/openvpn.php: Update server config if template has changed
  • lib/apps/openvpn.php: Fixed long standing bug with config tar package not being created properly (empty)
  • etc/seedbox/config/root.cron: checkDirectories changed to hourly. Add @reboot tasks for checkInstances, checkLighttpdInstances and checkDirectories and have some delays for the @reboot tasks so not everything launches on the same second.
  • soft.sh: do not delete scripts directory - usually nothing is removed from here, just added or updated. Removed web proxy removal as we have not had that many years in any case
  • util/setupNetwork.php: Added logging tcpsack
  • util/update-step2.php: Put date into /var/run/pmss/updated for reference, also make sure dir exists.
  • cron/checkGui.php: New cronjob, every 10 mins to check if index.php is intact, copy new one if not. This can occasionally happen when user is over quota and beyond burst period.
  • etc/skel/www: index.php, info.php, stats.php, welcome.php updated to the latest being distributed.

29/11/2019

  • util/setupNetwork.php: Add iptables rule to drop certain size mss, mitigating tcpsack attacks; and further disable tcp_sack kernel
  • util/setupNetwork.php: Fix not clearing INPUT rules first, thus making multiples of the same rules.
  • util/setupNetwork.php: Dropping specific targets now use insert (to make high on list) and accept rules at bottom
  • root.cron: Long standing bug with @reboot setupNetwork cron job fixed. This was causing why OpenVPN does not always function as expected.

21/11/2019

  • ruTorrent: remove diskspace plugin, and include our own disk quota plugin

08/11/2019

  • cron/trafficLimits.php: Add unix socket support
  • cron/trafficLimits.php: Check & create runtime directory if does not exist
  • cron/trafficLimits.php: Issue second traffic limit removal command to make sure it actually happens
  • ~/.rtorrentExecute.php: Drop waiting periods to make restart happen faster
  • root.cron: Drop checkinstance periods from 5 minutes to 2 minutes for faster restarts
  • root.cron: Redirect checkDirectories log output to /dev/null. Cron may not execute if redirected to log directory and log directory does not execute ... making circular issue
  • root.cron: Drop traffic stats collection from 5 mins to 10mins
  • cron/trafficStats.php: Save daily traffic consumption to ~/.trafficData
  • www/info.php: Add chartjs loading
  • www/stats.php: Add daily charts for traffic consumption
  • util/update-step2.php: Install sox, nzbget
  • util/userConfig.php: Removing some commented out code
  • RuTorrent updated to latest V3.9 stable release! Not updating existing users yet, only for userspace rebuilds & new users

06/10/2019

  • config/root.cron: Typoes in the renice commands fixed.

28/08/2019

  • lib/rtorrentConfig.php: added file exists check and creation in the write config function, in case user has deleted their config file

06/08/2019

  • cron/trafficStats.php: Check runtime directory readability and change way of creating to via shell
  • fix localnet filepermissions on rtorrent conf
  • change rtorrent config template scgi settings
  • rtorrent config chmods localnet file to 644 to avoid random permission issues
  • change rutorrent config scgi settings

30/04/2019

  • apps/packages.php had a typo this has been fixed.
  • apps/rclone.php: bug fixed version checking, every other update would remove rclone completely etc.

29/04/2019

  • Change jessie repos again, and disable valid until verification for repos to continue functioning as expected
  • refactored a lot of apt package install to lib/apps/packages.php
  • refactored a lot of package install from install.sh to lib/apps/packages.php
  • refactored more python stuff to lib/apps/python.php
  • Some update-step2 refactoring on useless legacy stuff, remove api remotekey etc.
  • Refactored runtime directories checking from update-step2 as a new cronjob - further found potential security issue where a local user could after reboot gain priviledges to the runtime data in /var/log/pmss or logs /var/log/pmss

23/04/2019

  • Update jessie repos
  • Rework sonarr/nzbdrone installer, staging due to jessie repo changes was a necessity
  • create /scripts/lib/apps folder to refactor different application installers for easier maintenance
  • lib/apps: did megatools, mono, sonarr installers, btsync
  • Added resilio sync THIS IS UNTESTED 100%
  • Renamed btsync to be btsync1.4 and created symlink btsync to point to 2.2 WARNING: If you are running btsync 1.4 this might break you
  • cron/trafficStats.php: Make sure runtime temp directory exists. Sometimes after reboot this folder might be missing for few hours
  • update.php: Remove apt-get update, move it to update-step2. comment out setupApiKey as that is not currently being utilized
  • update-step2.php: Remove debian lenny and squeeze repos etc. as those have not been used for years in production. There might still be some wheezy installations in the wild so not touching those.
  • update-step2.php: For some bizarre reason there was apt-get install for rar, unrar and mktorrent in the repo configs, removed those but made sure those are installed quickly after
  • lib/apps/syncthing.php: Syncthing installer THIS IS UNTESTED 100% it may or may not work. Probably does work.
  • lib/apps/openvpn.php: Move it here, some refactoring
  • lib/apps/rclone.php: Move installation here, some refactoring and commenting for easier future updates. Version updated to 1.47 let us know if there is regressions due to update
  • lib/apps/vnstat.php: Move install + config here, move various spread around bits regarding deb8 conflicts to here as well.
  • update-step2.php: More commenting and deb version peculiarity fix refactoring and removing more support bits for deb5+6
  • lib/apps/python.php: Beginnings of this nightmare. Only added youtube_dl pip3 install so far.
  • util/makeMonitoringRules + cron/trafficLog: localnet file was using wrong path, fixed it to be /etc/seedbox/config/localnet
  • update-step2: move above mentioned localnet file during update
  • setupSkelPermissions: set localnet file permission to globally readable - yes i know wrong place but there was some openvpn configs already here too
  • lib/rtorrentConfig.php: Add localnet preferred ipv4 filtering if file is present

04/01/2019

  • util/update-step2.php: Install spidermonkey JS engine for pyload use, packages

Changes 2018

11/12/2018

  • seedbox/config/root.cron: Add ionice classification for cp as idle process
  • seedbox/config/root.cron: Add renice & ionice classification for ffmpeg and rclone as lowest priority
  • seedbox/config/root.cron: Ever so slightly bump up the ionice classification for rtorrent processes
  • seedbox/config/root.cron: Increase process checking frequencies for faster restarts etc.
  • util/update-step2.php: Added package 'ranger' to be installed, a ncurses frontend filemanager.
  • util/update-step2.php: Install nethack ... Because nethack! ;)


02/10/2018

  • util/setupSkelPermissions.php: Also sets openvpn config dir perms. TODO refactor, rename etc.
  • util/update-step2.php: put openvpn config to the file update list
  • util/update-step2.php: change filemanger recycle bin permissions to 771
  • util/update-step2.php: re-enable creating openvpn-config.tgz when making the openvpn config the first time

21/09/2018

  • util/update-step2.php: Install Filebot 4.8.2

17/09/2018

  • Release was actually a dev version and latest updates were not pushed, this has been fixed. Caused myriad of network accounting issues etc. weird ones.

10/09/2018

  • proftpdTemplate: Add directive PassivePorts 60000-65535 to always have a known range

06/09/2018

  • util/setupNetwork.php: Bogon/Martian drop filters. Base groundwork to add a global bad actor banning mechanism (ie. bruteforce attempts)
  • util/update-step2.php: Install package ipset
  • util/update-step2.php: Compile & install iprange
  • util/update-step2.php: Compile & install firehol
  • etc/seedbox/config/template.nginx-user: Increase limit rates to: 500MiB full speed, after that 4096KiBps for users. Public remains as low as before.
  • etc/seedbox/config/template.nginx-proxy_params: buffering disable
  • etc/seedbox/config/template.lighttpd: Increase server max rate to 51200KiBps, and per connection to 16386KiBps, thus allowing nice bursting at beginning of a transfer :)
  • util/update-step2.php: remove some very ancient resolver updates
  • util/update-step2.php: remove a deprecated testfile removal
  • util/setupNetwork.php: Updated link speed parsing
  • util/setupNetwork.php: Determine our net link for getting speed with ethtool

03/09/2018

  • util/makeMonitoringRules.php: Trim localnets file
  • cron/trafficLog.php: Trim localnets file
  • cron/trafficLog.php: Fix grep clauses and a few other things related to the localnets monitoring
  • showTraffic: Remove each line "Traffic", kinda redundant. making it easier to read :)
  • cron/trafficStats.php: Logging date typo fix, even heavier parse error message suppression.
  • config/root.cron: Increase trafficStats cron to every 5 minutes as well, and wait 20secs for logging to complete
  • cron/trafficLog.php: More grep clause fixes, data collection fixes etc. in relation to local networks. Was not saving general data usage.
  • showTraffic: Change when to go for larger measurement unit for display purposes.
  • lib/traffic.php: Save local data consumption as well.
  • util/remote/userTraffic.php: Change output to support "normal" and "local" data consumption metrics

02/09/2018

  • showTraffic: make username field wider and shorten if -local file being displayed.
  • showTraffic: some display output formatting fixes
  • cron/trafficStats.php: notice of parse error supression was in wrong place
  • cron/trafficLog.php: Change to 1250MB/s max network utilization for 5minute average to enable monitoring. Still #TODO use ethtool to determine this
  • cron/trafficLog.php: log local networks separately on -localnet files
  • cron/trafficLog.php: Bug fix unmatched traffic metering (wrong file)
  • util/makeMonitoringRules.php: Separate local networks traffic monitoring per user, and finally add a "unmatched owner" rule for local networks
  • util/makeMonitoringRules.php: formatting fixes
  • New optional config file: /etc/seedbox/localnet, you can input your local networks here with same annotation as iptables. One per line

20/08/2018

  • util/update-step2.php: Some typo fixes
  • util/userTransfer.php: Add public web directory and homedir rutorrent settings folder into migration
  • util/update-step2.php: Add package python-virtualenv to be installed

18/08/2018

  • util/update-step2.php: Install gdrivefs, or atleast try ...
  • util/update-step2.php: remove python-pip apt package, easy_install it and link properly. This hopefully fixes python pip issues more than creates new issues, but the whole repo is horribly horribly broken with constantly stuff breaking.
  • util/update-step2.php: Add unionfs-fuse, sshfs and s3fs packages.

12/08/2018

  • util/update-step2.php: Fix flexget installation

11/08/2018

  • util/ftpConfig.php: Template file changed for 16 connectios per user & host max, plus max instances raised to 60 from 30. No changes to that file, just the template

10/07/2018

  • util/update-step2.php: Updated rclone to version 1.42. Old download link was not functioning anymore, so moved hosting to our own server.

11/06/2018

  • util/userTransfer.php: Disable compression for transfers. Most data is anyways compressed, so this ensures higher data rates.
  • util/update-step2.php: Add installation of genisoimage xorriso

Changes 2017

28/07/2017

  • cron/trafficStats.php: Fix date formatting for output
  • cron/trafficStats.php: Limit parse error messages + make supression if too many (50 for single user)

26/07/2017

  • rclone update to v1.37
  • New rtorrent.rc template file, with some new options enabled

19/07/2017

  • Rename template.nginx to template.nginx-sites-default
  • Add template.nginx-conf for /etc/nginx/nginx.conf
  • template.nginx-conf has client_max_body_size 4096M -> Fixes the 1MB file upload size limit which crept back in earlier
  • Fix update-step2.php a debian version selector, for which the earlier debug code was for
  • Update util/createNginxConfig.php to match above templating changes

18/07/2017

  • Added packages htop mtr aria2 to be installed by default
  • Deb8 specific package setups regarding git and znc debug + extra apt-get instructions to attempt get git installing finally
  • Add also systemctl instruction for lighttpd not to start

12/07/2017

  • Yet another vnstat fix (permissions)
  • Fix rclone install path

13/06/2017

  • cron/cpuStat.php: Collect CPU usage stats, user, nice, sys and idle for 2mins. Saves to /var/run and http publicly fetchable serialized output
  • config/root.cron: Add cpuStat to run every 15mins, after 45sec delay (avoid other scheduled cronjobs this way)

25/03/2017

  • util/update-step2.php: Hack for nzbdrone & mono to get them installed on a Debian 8 system
  • util/update-step2.php: Deb8: get prepackaged easy-rsa and configure openvpn, make path issue symlinks to avoid deb8 issues
  • util/update-step2.php: revert back to deb7 specific openvpn config to be the normal case and add selection for differences between deb8/deb7.
  • util/update-step2.php: remove deb8 speficic openvpn rsa config
  • rtorrent.rc min peers setting increased slightly
  • lib/rtorrentConfig.php: revamp port selection method

11/03/2017

  • util/update-step2.php: Refactoring on git, znc, python3, acd_cli installation (deb8 specific) in an attempt to figure out Deb8 issues regarding these (part installed, part not!)
  • util/update-step2.php: added lighttpd to list of daemons not to be run as system (only per user). TODO still is to check runlevel management changes for Deb8 (many repo pkgs has not been updated neither)

07/01/2017

  • cron/cgroup.php: Do not execute currently, not complete
  • cron/diskIostat.php: Some commenting and slight refactor
  • cron/trafficStats.php: refactor, commenting, couple typo fixes
  • cron/checkInstances.php: Slight refactoring, some commeting and clarifying
  • cron/checkQuotas.php: remove API calls, that method is deprecated
  • cron/trafficLog.php: adding a few code comments, increase temp file entropy


Changes 2016

29/12/2016

  • skel/rutorrent/plugins/autools/plugin.info: add to info note for watch dir breaking things
  • plugins/autotools/lang: added warnings to en, de, fi, es
  • install.sh: add zip pkg
  • install.sh: add -j make parameter
  • util/update-step2.php: add -j make parameter
  • util/update-step2.php: add zip pkg
  • addUser.php: comment out retracker stuff

07/12/2016

  • util/update-step2.php: (Re)install git, some updates on pkg management on deb8
  • util/update-step2.php: Fix vnstat not updating on deb8
  • util/update-step2.php: Install Acdcli for amazon cloud drive access
  • cron/diskIostat.php: fail if no serial block devices found
  • cron/diskSmart.php: Added to collect smart data of disk drives

27/10/2016

  • cron/diskIostat.php: Include server timestamp on the results
  • config/root.cron: Collect IOStat results every 5mins instead of 15mins (2min average)

01/10/2016

  • util/update-step2.php: Add pigz (parallel implementation of gzip, [1]) to be installed
  • util/update-step2.php: Small refactoring on apt package installs

25/09/2016

  • lib/update.php: ruTorrent config add setting top directory correctly, filemanager plugin for example requires this

08/09/2016

  • util/update-step2.php: Add Jessie (Deb8) repors
  • util/update-step2.php: Add OpenVPN config for Jessie, OpenVPN config now distro specific
  • util/update-step2.php: Remove some old (deb5/6) apt-get instructions
  • util/update-step2.php: Debian 8 ffmpeg install from backports added
  • util/update-step2.php: znc package selection for Deb8


05/09/2016

Preliminary support for deb8, most things work.

  • install.sh: Lots small changes to add support for deb8
  • util/update-step2.php: Added headers to launch from cli

03/09/2016

  • util/update-step2.php: Install funcsigs pip package, slight update to upgrade list of packages for flexget

03/08/2016

  • Small visual updates

25/07/2016

  • util/update-step2.php: Removed some apt-get instructions as they are in install.sh already
  • install.sh: Some slight refactoring on the package installs
  • util/update-step2.php: Install rclone
  • cron/diskIostat.php: Make iostat current stats available also via direct http download

21/02/2016

  • checkInstances.php: Typo fix
  • checkLighttpdInstances.php: Typo fix

07/02/2016

  • terminateUser: Comments, multiple killall and userdel directives as procs sometimes just don't want to die ... or are being relaunced. TODO check why this actually happens instead of this hack
  • cron/checkInstances.php: If user is suspended, skip trying to start rTorrent ... Infact kill all procs :)
  • cron/checkLighttpdInstances.php: Same thing here

28/01/2016

  • cron/trafficLimits: Some refactoring, adjusting limits for new speed caps and added code to choose limit level based on total traffic limit as well (512K/s allowed to do more than limit on minimal plans)
  • lib/rtorrentXmlrpc: Add function for setting download rate
  • cron/trafficLimits: Restrict download speed as well IF entry plan (less than 500GiB traffic limit) to 20Mbps

24/01/2016

  • update-step2: Install libffi-dev, python-dev, pyopenssl ndg-httpsclient pyasn1 cryptography for better flexget support
  • update-step2: replace some nameservers with sed to OpenDNS (Online.net is censoring some trackers)

14/01/2016

  • update-step2: Removed creating /etc/autodl.cfg, instead removing it, as it's not a prerequisite for autodl anymore and we can allow all permissions.

11/01/2016

  • addUser.php: Set also user homedir .trafficLimit file (so we can display it in GUI), slight refactor on this section
  • util/userTrafficLimit.php: Same thing as above
  • (GUI) Stats: Added traffic limit number
  • (GUI) Welcome page: Added meter for traffic limit

05/01/2016

  • util/update-step2: Initially typoed filename as tesfile, remove that and create the correct file
  • util/update-step2: Create /var/www/testfile, 100M random data

01/01/2016

  • cron/diskIostat: Added history logging raw output (We are seeing IOPS outputs of 1000+ from magnetic drives, so need debug data for that)
  • cron/diskIostat: History logging wrong output mode (should've been append).

Changes 2015

30/12/2015

  • update-step2: Refactoring the openvpn setup section, which also bug fixes the situation where openvpn is setup but configs for users were not made

28/12/2015

  • upate-step2: Bug fix: cgroup-bin apt-get did not have autoconfirm, requiring human intervention to complete update

26/12/2015

  • cron/trafficLog: Log www-data & unmatched traffic as well
  • cron/trafficStats: Parse for all log files found on logging dir, so includes unmatched, www-data, terminated users etc.
  • cron/trafficStats: Log total data for the month period
  • cron/trafficStats: Removing legacy code, small refactoring and commenting bits
  • showTraffic: Show for all data files found
  • lib/traffic.php: saveUserTraffic now saves to /var/run/pmss/trafficStats, and creates that directory if it does not exist
  • lib/traffic.php: Bug fix: Skipped if data exceeded 7500Mb in 5mins... that's only 25MB/s. Now limit is 500MB/s with 150 000MiB check.
  • showTraffic: Now uses /var data files. Instead of skipping on missing, throw an error
  • showTraffic: If data is 0 for the month, skip displaying this user (gone already)
  • util/makeMonitoringRules.php: Add 'www-data' to list of users
  • showTraffic: Display month total across all users in TiB
  • util/update-step2: Fix vnstat configuration typo to set the maximum link speed to 1Gbps
  • cron/cgroup.php: Created, initial version. Needs quite a bit of refactoring, testing level right now.
  • util/update-step2: install cgroup-bin, add fstab cgroup entry and mount it
  • etc/seedbox/config/root.cron: Add cgroup cron job, execute once per hour


21/12/2015

  • update-step2: run apt-get upgrade at the end

20/12/2015

  • update-step2: Install btsync 2.2, as btsync2.2. System will have 1.4 as btsync and 2.2 as btsync2.2

19/12/2015

  • update-step2: separate setup apt sources.list.d/sonarr.list. This also means Sonarr will now update normally.
  • createNginxConfig.php: Now always run lighttpd config. This is a bug fix, where manually changed lighttpd port would eventually mismatch on nginx
  • configureLighttpd.php: Increased potential port range.
  • skel/.lighttpd/custom: Empty file created to avoid warnings. This is where users can input their custom lighttpd settings if they need some.
  • userTransfer.php: Added .lighttpd/custom for migration
  • userTransfer.php: Decreased the time for re-attempting user migration
  • recreateUser.php: Also copy htpasswd file from old version

08/12/2015

  • (GUI) Change welcome page quota display to use Kibibyte format to avoid confusion.

30/11/2015

A privileged user could use an internet download manager to basicly DOS the server with huge amount of threads trying to download large files (gigs in size), with sufficient thread quantity eventually linux kernel couldn't schedule any I/O time for disk nor ram for essential applications such as SSH. This could not be done by unprivileged user since requirement is very large file sizes. This would lead to a race condition as nginx wants to cache as fast as possible, hence "downloading" from user lighttpd process as fast as possible, which results in heavy RAM consumption and 'especially' heavy disk I/O activity - with sufficient threads scheduling breaks down.

  • template.lighttpd: Introduce per connection rate limit (2048k/s) and global rate limit (12800k/s)
  • template.nginx-user: Introduce user con limit to 16, rate limit (2048k/s) after 100MiB
  • template.nginx-user: Introduce public con limit to 8, rate limit (512k/s) after 100MiB
  • tmeplate.nginx: Add shared memory zone for connection limits

18/11/2015

  • util/userConfig.php: Lower limit on filesLimit for very small disk quotas, set to 15000
  • (GUI) welcome.php: Display of /etc/seedbox/config/vendorWelcome moved to top of the welcome page

12/11/2015

  • cron: renice mono and ionice mktorrent added
  • update-step2: install some perl modules

26/10/2015

  • .lighttpd.conf: Add include_shell for .lighttpd/custom for user custom settings
  • .lighttpd.conf: Add rutorrent no cache headers, and also add mod_setenv to be loaded
  • rtorrent.rc template: Increased base buffers

24/10/2015

  • rutorrent/plugins/hddquota: Bumped version to 1.15
  • ruTorrent: Updated to version 3.7
  • install.sh: Bumped rTorrent/libTorrent to 0.9.6/0.13.6, bumped xmlrpc-c to revision 2776
  • rTorrent: Updated to 0.9.6/0.13.6

16/10/2015

  • update-step2: Make openvpn config package even if this is not an install run
  • update-step2: fix flexget install if clause

12/10/2015

  • pulsedBox: Login for mds now fixed
  • ruTorrent: Changed URI for css & js files to make sure cached copies are not attempted with new version
  • update-step2: put back in to update ruTorrent index.html
  • update-step2: Configure OpenVPN, Create certs, create client config package and provide files to users at /home/openvpn-**servername**.ovpn and /home/openvpn-**servername**.crt where as dots (.) have been replaced with dashes (-)
  • update-step2: Make a tarball of OpenVPN config in skel, and provide it to userspace as well.
  • setupNetwork.php: Create OpenVPN masquerade rules
  • (GUI) welcome.php: Add beta disclaimer to ownCloud
  • update-step2: Oops, openvpn config should probably reside under www
  • update-step2: trim hostname before using for filename
  • update-step2: flexget installation string match fix


11/10/2015

  • update-step2: Install mkvtoolnix, add mkvtoolnix repo
  • userConfig: Refactoring on ruTorrent stuff
  • update-step2: Refactoring on ruTorrent stuff
  • skel/www/rutorrent: Updated to latest with new plugins
  • etc/seedbox/config/template.rutorrent.config: Updated to new
  • etc/seedbox/config/template.rutorrent.access: Updated to new
  • lib/update: Add rutorrent config function
  • userConfig: Change to using that function
  • update-step2: Reconfigure ruTorrent
  • update-step2: Reordering all together everything related to ruTorrent on per user basis and some refactoring
  • ruTorrent Update Latest version, should migrate settings etc. and reconfig properly :)
  • update-step2: removal of removing check port plugin. Seems to be working ok again?

10/10/2015

  • update-step2: Install flexget, buildtools32, python-pip

09/10/2015

  • install.sh: Slight reordering of package installs to make initial setup even faster (human time wise)
  • install.sh: Add lftp to be installed
  • update-step2: Add lftp to be installed

04/10/2015

  • php.ini: Default memory max increased to 512MiB, upload max filesize to 16GiB and max simultaneous uploads to 50

02/10/2015

  • addUser.php: refactoring and adding a chown to make sure everything is setup allright in regards of retrackers

01/10/2015

  • update-step2: Another fix for retrackers related stuff
  • userPermissions.php: Couple small fixes related to the above

30/09/2015

  • Small update to rTorrent + libtorrent, faster tracker check interval default setting, and close idle connections faster
  • install.sh: Refactoring to have all questions at the beginning
  • install.sh: Added many of the pkgs from update-step2 here, so we can later on remove them from update-step2
  • util/setupNetwork.php: commented out link speed determination - not required
  • install.sh: sysfs added net.ipv4.ip_forward = 1
  • setupNetwork.php: added enabling ipv4 ip forward
  • userPermissions.php: path fixes
  • addUser.php: Typo fix
  • (GUI) welcome.php: Updated with owncloud install link, and removed the need to visit setup-owncloud-finish.php
  • (GUI) Frames: Added owncloud tab if present
  • skel/www/setup-owncloud-finish.php: removed
  • update-step2: removed setup-owncloud-finish.php, and made owncloud setup file insertion conditional
  • (GUI) welcome.php: Organized the download links and fixed them :)
  • (GUI) welcome.php: Fixed quota warning var giving notices on error log
  • (GUI) stats.php: Changed getting IP to remote service

29/09/2015

  • update-step2, addUser.php, userPermissions.php: Continuum of the earlier, apparently there's a slight logic error on ruTorrent preventing torrents directory from being created

28/09/2015

  • update-step2: Fixed permissions bug introduced by yesterday's changes
  • addUser.php: Same fix as above
  • userPermissions.php: Fixing retroactively these. This is executed on server update.

27/09/2015

  • checkInstances: Killing possible rTorrent processes fix.
  • root cron: Change check Lighttpd for every 5 mins, delay beginning 30 secs
  • error-502: Wording change
  • addUser & update-step2: Adding retracker plugin config

04/09/2015

  • createNginxConfig: Trim port number added
  • configureLighttpd: Increased potential port range
  • terminateUser: Remove lighttpd port config and nginx config
  • terminateUser: Some wording changes shown to user.

30/08/2015

  • Nginx: Custom 502 error page
  • util/createNginxConfig.php: Add template for proxy_params
  • Nginx: Increase proxy read timeout to 120s
  • skel/.lighttpd/php.ini: Increase max execution time to 90s and increase max memory to 256M per process! Increase Max file upload size from 2M to 32M, POST max size from 8MB to 32MB
  • update-step2: Fix owncloud setup files path.
  • update-step2: Add php.ini to the files to be updated
  • update-step2: Install packages flac, lame, lame-doc and mp3diags


11/08/2015

  • cron/diskIostat.php: Add disk quantity
  • cron/diskIostat.php: fix output camelCase
  • Added ownCloud installers - Install is now very simple and easy!
  • template.lighttpd: Added ownCloud directives for security
  • util/update-step2.php: Added update-rc.d lighttpd remove
  • util/update-step2.php: Added copying latest ownCloud setup files
  • cron/diskIostat.php: bug fix on device selection, on some edge cases the grouping was chosen incorrectly

10/08/2015

  • cron/diskIostat.php: Added cron job to gather current iostat info, 120 sec data
  • cron.d/pmss: Run diskIostat every 15 mins
  • util/userPermissions.php: Fixed xargs argument bug for delimiter
  • cron/diskIostat.php: Added logging to -history file as well for historic data, for potential later needs

02/06/2015

  • userPermissions.php: Find all directories under user account and chmod them to 751 (default directory permissions: rwxr-x--x).

06/04/2015

  • userPermissions.php: Chmod .ssh if the directory exists

05/04/2015

  • userPermissions.php: Chowning as well to make sure right user & group owns the directory.
  • updates-step2.php: Added ZNC and ZNC-EXTRA packages

02/04/2015

  • configureLighttpd.php: Typo fix

01/04/2015

Some bug fixing to yesterday's release

  • userPermissions.php: chmod 771 couple temp dirs for each user's lighttpd
  • update-step2.php: apt-get remove go git mercurial added since these packages are not needed


31/03/2015

This release is all about converting to per user lighttpd processes and several bug fixes, and hence enhancing service quality and security while opening the door for future new feature developments. This also adds capability for public web hosting from the seedbox.

  • install.sh: remove packages golang git mercurial
  • update-step2: remove packages golang git mercurial
  • install.sh: also put into fstab vfsv1 instead of vfsv0
  • update-step2: remove migration to vfsv1 since this is prone to breaking quota data. need to look for better migration method.
  • etc/seedbox/config/template.lighttpd added and created
  • etc/skel/lighttpd directory added
  • etc/seedbox/config/template.nginx-user added and created
  • etc/seedbox/config/template.nginx added and created
  • util/configureLighttpd.php: Created configurator for per user lighttpd configs
  • etc/skel/lighttpd/compress directory added
  • etc/skel/lighttpd/upload directory added
  • template.lighttpd: both password protected (normal GUI) and public web directories included in the config where user can run whatever they please.
  • changePw.php: changes now the per user htpasswd file (username/.lighttpd/.htpasswd)
  • addUser.php: use changePw.php script instead of set in the file
  • addUser.php: if password rand has been passed, use a random password.
  • addUser.php: remove chmod rules
  • addUser.php: if runtime/trafficLimits dir does not exist, create it
  • addUser.php: use configureLighttpd.php script instead of setting it up hardcoded within the script
  • addUser.php: change expire date to 2100-01-01
  • addUser.php: Do not add user to www-data group
  • util/createNginxConfig.php: Create per user proxy config files (idempotent) and set default nginx config from template
  • createNginxConfig: Create SSL config.
  • template.nginx: Include SSL config
  • scripts/startLighttpd added and created
  • scripts/cron/checkLighttpdInstances.php added and created
  • update-step2: Added all the lighttpd configuring, installing packages, managing processes etc. there. This should be the bulk of conversion
  • addUser.php: create nginx config, and restart nginx
  • config/root.cron: Comment out checkHttpd for now and add checkLighttpdInstances in there
  • userConfig.php: Do not launch traffic limit script if none has been passed
  • lib/rtorrentConfig.php: If no logger, do not echo the contents (rel: addUser.php needlessly echoing rtorrent config)
  • changePw.php: if password defined it's not set to correct variable and thus cannot set password
  • changePw.php: Also check for existence of per user htpasswd file!
  • util/checkUserHtpasswd.php: Created to move the user's password from global to per user.
  • update-step2: run checkUserHtpasswd.php and restart all lighttpd processes
  • update-step2: lots of permissions updates
  • update-step2: remove some deprecated, commented out code
  • util/userPermissions.php: created this to set all the folder and file permissions for the user directory. Now they are in one place!
  • addUser.php: use userPermissions.php
  • update-step2: use userPermissions.php
  • template.lighttpd: Use per user php.ini and change fcgi settings a bit
  • skel/.lighttpd/php.ini added
  • configureLighttpd.php: Add .lighttpd dir if it doesn't exist yet, separately check for php.ini as well.
  • update-step2: Do not remove autotools plugin if it is present
  • update-step2: add package ntp
  • checkUserHtpasswd.php: Chown the newly created .htpasswd file
  • userPermissions.php: set htpasswd file perms as well
  • update-step2: move earlier in the process to make the move to nginx etc. for working conversions. make sure lighttpd processes are stopped.
  • update.php: remove recreateLighttpdConfig & restart, replace with nginx
  • release packager: include .lighttpd directory into the skel package.
  • userPermissions.php: some chowns and extra rules required for smooth conversion
  • terminateUser.php: replace lighttpd related commands with nginx replacements
  • terminateUser.php: remove deleting htpasswd. Add deleting home dir in case userdel does not remove it
  • recreateUser.php: Changed to fit the new model.
  • configureLighttpd.php: Also check for public html directory
  • update-step2: Do the configs twice to be certain, as another update step might have affected.

07/03/2015

  • quotaFix.php: added -n flag for quotacheck to automatically answer questions
  • install.sh: Update quota vfsv0 to vfsv1 to support large quotas
  • update-step2: Update quota vfsv0 to vfsv1 to support large quotas, and commit change + execute quotaFix
  • update-step2: Check if user is suspended, if so do not make updates
  • update-step2: Reinstalling unpack plugin for ruTorrent which was earlier removed for abuse causing extremely high loads on a server

27/02/2015

  • update-step2: moved the location of nzbdrone repos to the normal repo list, cannot move sonarr because https transport needs to be first installed.
  • update-step2: Refactored stopping & disabling services not needed for "global" user
  • update-step2: added apt-get -f install -y to fix potentially broken or missing dependencies
  • update-step2: Forgot to add -y toggle for sabnzbdplus package install requiring manual intervention

26/02/2015

  • install.sh: Added locate apt package, since some minimal installs are missing this
  • update-step2: Added locate apt package
  • install.sh: Added python-pycurl, python-crypto, python-central packages
  • update-step2: Install pyLoad and dependencies python-pycurl, python-crypto, python-central, golang, git, mercurial
  • install.sh & update-step2: Install packages gcc g++ gettext python-cheetah curl fuse glib-networking libglib2.0-dev libfuse-dev apt-transport-https
  • update-step2: Compile & install Megatools from source
  • update-step2: Re-ordered package installs as some installation & compilations were attempted BEFORE installing the dependiences.
  • update-step2: get, compile and install Mono 3.8.0
  • update-step2: Install Sonarr / Nzbdrone. To start it execute mono /opt/NzbDrone/NzbDrone.exe and then browse to http://servername.pulsedmedia.com:8989 - remember to change port and set security + authentication credentials!
  • update-step2: To make sure they are not running for non-specific seedbox user, stop btsync and pyload, and update-rc.d disable them
  • update-step2: Install sabnzbdplus

02/02/2015

  • update-step2: ntpdate for some reason was not installed by default - included this on update
  • update-step2: now vnstat config is checked on every update
  • update-step2: vnstatConfig also sets maxbandwidth higher
  • apt-get clean & update for some reason missing from update.php - added, and also temporarily for step2

31/01/2015

  • Security update on lighttpd config.

30/01/2015

  • btsync sample config now in skel as .btsync.conf-sample
  • btsync binary added to /usr/bin/btsync now.

21/01/2015

  • rutorrent/share/settings/rss folder was missing, created to skel and added to update for creating it

19/01/2015

  • trafficStats: Added logging also date & time. For some reason this was not logged.
  • trafficLog: removed serverApi calls as cnc is not utilized anymore
  • skel/www/rutorrent/plugins/check_port removed as well. was prior removed only on update. this plugin doesn't function anymore as expected.
  • some debug logging added for trafficStats cron


Changes 2011-2014

Can be found at PM Software Stack Changelog 2011-2014

Retrieved from "https://wiki.pulsedmedia.com/index.php?title=PM_Software_Stack&oldid=1002585"