PM Software Stack

From Pulsed Media Wiki
Revision as of 16:56, 10 February 2012 by Nucode (talk | contribs)

PMSS, short of PM Software Stack is a compilation of scripts to manage single server side seedbox configuration, turning instantly and conveniently an regular Debian 5.0 server into a seedbox. The PMSS does not contain multiserver management utilities nor clustering support out-of-box, but does include PM Master GUI remote update features. Debian 6.0 squeeze support is still in unstable status but several production servers already do exist. PMSS currently only supports rTorrent+ruTorrent stack.

General features


Using PMSS

See Category:PM Software Stack Guides

Changelog

Changes 2012

10/02/2012

  • Fixed torrent creation from ruTorrent
  • Fixed quota plugin (JSON parser has been changed to strict parsing)
  • Updated quota plugin version information
  • Fixed ajax manager recycle bin
  • Made ruTorrent to update browser cache
  • Created framework for easy user space files updating during server update (files dependant upon server upgrade)
  • Removed recreateLighttpdConfig + lighttpd restart from update step 2 (should by now be in main update in 99.9%+ servers)
  • Changed update.php to do lighttpd force-reload instead of restart
  • Refactored update-step2 slightly: Moved specific files, and user specific stuff into single loop
  • Added several packages to be installed during server installation, which were only on update. They still remain on update as well before better solution can be done (dselect)

09/02/2012

  • Reversing yesterday's update due to security concerns
  • Removed installer speedups due to stability concerns

08/02/2012

  • Fixed typo in install script preventing quota from being compiled

05/02/2012

  • Lighttpd session "stability" upgrade, fixing issue causing session timeouts/re-requiring authentication

04/02/2012

  • Added mediainfo
  • Added server root document to redirect to pulsedmedia.com
  • Added php5-geoip extension to be installed
  • Simplified update-step2 scripts being installed :)
  • Removed apt dist-upgrade from install
  • Simplified (less) apt-get lines during install
  • Reordered apt-get commands during install for swifter installation
  • Removed some "junk" code from install
  • Updated copyright text in install.sh
  • Added some information on header of install.sh
  • Install.sh some additional comments
  • Made installation slightly quicker by usage of background processes (BETA, if your disk is extremely slow this may cause installation to fail)
  • Reordering other items in install.sh to make installation faster
  • Some preliminary work for supporting whitelabeling
  • Removed umask, hash interval, hash max tries rtorrent configures (deprecated)
  • Updated rTorrent + ruTorrent to versions 0.9.0/0.13.0

Note: Please report any issues due to the upgrade of rTorrent immediately to Pulsed Media. The patch has been tested on a small group of servers, but all situations are hard to foresee.

31/01/2012

  • Fixed some typos in update-step2

28/01/2012

  • Added iotop and cfv to be installed from apt repo.
  • Added -m toggle for htpasswd because for some reason this is not default in Debian Lenny
  • Added SSL support (self signed cert), just change http to https
  • Added Webdav support, accessible from "/webdav-USERNAME/" *** BETA: Permission issues, your mileage will vary
  • Removed SSL key creation from install
  • Update-step2 creates SSL key automatically utilizing the server hostname, removes old deprecated key

Changes 2011

15/12/2011

  • Added .irssi/config to user dirs, autojoining to #PulsedMedia on Freenode when irssi is launched
  • Updated GUI IRC links, Chat tab and Adobe download links, with other small additional information and updates.
  • Removed Network Storage tab from GUI because Dropbox is blocking pageviews coming from our system for unknown reason.
  • Added Proxy information on GUI
  • Update-step2 sees if .irssi/config exists and if it does not copies it there

30/11/2011

  • Chosen rTorrent SCGI port range increased from 4k to 8k.
  • "fractional blocks" for config. Resource calcs are based on number of blocks, and fractions were not allowed prior.

29/11/2011

  • During update also update user crontabs. Added to both update.php and step2, to be removed from step2 after this update has seen significant adoption.
  • Restarting lighttpd during update, also done doubly for now
  • Moved setup api key to regular update script. In several months it will potentially be removed from update all together as it's a installation step, and included to update older nodes.
  • Scripts permissions update
  • updateQuotas.php -> cron to update all user quotas, as discreet user from central cron to ensure it gets updated.
  • Updated root & user crontabs to reflect updateQuotas

28/11/2011

  • quotaFix now removes aquota*new files to avoid collision and thus failed update via quotacheck
  • userTorrents.php : Lists the count of torrents each user has, for quicker analysis usage stats

23/11/2011

  • Permissions setup script added, which is utilized by recreateUser.
  • recreateUser.php fixes

22/11/2011

  • Created recreateUser.php which recreates home dir, preserving data+session if existing. ALPHA

19/11/2011

  • Fixed saving API Key.

17/11/2011

  • Suspend and Unsuspend scripts fixed
  • recreateLighttpd backup fixed
  • Added access format of: servername/user-username/, for example: localhost/user-johndoe/. Thus supporting non FQDN setups
  • addUser.php calls recreateLighttpd instead of just adding the hostname, this ensures lighttpd config is always up-to-date.
  • Added to both update steps recreateLighttpd. It will remain on step2 for the next ~6months to ensure every system out there will have the update update.php
  • Removed the default "testing" torrents
  • Upload slots global, by default 6 times the non-global upload slots. It is "forcefully" inserted even if template does not it have for the moment. this forcefull insert will be removed in the future.
  • checkInstances now also checks rtorrent configs, and if changed lists them on /root/changedConfigs for admin review

25/8/2011

Streamlined installer for faster setups :) First release of API, just one command now. Some things moved to BETA status and considered feature complete. Many bug fixes, additional commenting etc. also added

  • fixed userTransfer chowning lacking -R, thus UID was transferred for many files from the other server causing incorrect permissions
  • wrapped vnstat install into if !file_exists to see if vnstat is installed already.
  • added username 'srvadmin' to filter list on systemUsers
  • removed from installer editing munin.conf -> if used can be done manually. Interim solution.
  • put install vim as first thing on installer
  • setting node hostname straight after installing vim
  • moved setting up lighttpd as last thing
  • scripts, skel, etc are installed earlier in installation
  • SSL cert + quota as last items on setup
  • Executing update-step2 on installer (so you don't need to execute updater after setup to get everything in)
  • /scripts/util/setupRootCron.php -> creating root conf moved out of update.php
  • setupSkelPermissions.php, same reason as root cron setup
  • added setupRootCron and setupSkelPermissions to installer
  • Idempotent lighttpd configurator feature completed. Backups the config first.
  • Terminate user moved to beta status, as using lighttpd configurator now and restarting lighttpd.
  • Added for terminate second argument, if 2nd argument is --confirm, straight accepts termination without asking for approval, for making automation easier.
  • checkHttpd timer increased to 3 seconds before restarting lighttpd
  • terminateUser timer increased to 5 seconds after trying to kill all user processes
  • setupApiKey -> creates node key for API. Invoked upon setup.
  • added authorization key
  • removed /var/www/index.lighttpd.html
  • added /var/www/index.php redir to pulsedmedia.com
  • added beginnings of remote API for easier remote control management, by default in debug mode for now


8/6/2011

Bug fixes.

terminateUser.php:

  • Was not actually completing shell userdeletion
  • Added killing user processes (forgotten)
  • added sleeping 2 seconds to allow time for rTorrent/other processes by user to die
  • Added 2nd attempt to userdel after data removal
  • Fixed some typos in texts
  • Added check for user actually exists

NOTE: Still Alpha, non feature complete and not recommended for production use.

  • checkHttpd.php added sleep between lighttpd kill and restart, to avoid PID file mismatch scenario

update-step2.php:

  • First time being used
  • update.php moved cron & ftp config stuff AFTER step2
  • Install vnstat + vim (vim not by default on Debian 6)
  • Setup interface for vnstat
  • Configure vnstat for Debian 6
  • Make vnstat cron jobs for Debian 5: twice a minute update (thus no support for 1GigE interface)
  • Debian 5 add cron jobs to root.cron.additional config, create if needed

31/5/2011

  • GUI welcome page checking if proper quota information was passed has been added.
  • GUI welcome page malformed serialized data fix
  • GUI welcome page quota percentage missing % char
  • GUI welcome page quota text styles: White and location

30/5/2011

  • GUI update script quota checking added (doesn't try to update when user is over quota burst limit, no more blanking GUI when update happens!)
  • Users default quota information update rate has been increased to 10mins
  • HDD Quota plugin commenting added
  • HDD Quota plugin parser bug with bursted amounts fixed
  • HDD Quota now shows USED/QUOTA instead of FREE/QUOTA. Makes more sense.
  • HDD Quota "title" fixed (showing quota info) to be whole area of the meter
  • Welcome page quota meter added :) Basic graph, larger and constantly visible text for quota info :)
  • Basic idempotent lighttpd configurator (alpha, not suitable for production, TESTING only)

23/5/2011

  • terminateUser.php http password deletion fix, text fixes, executability.
  • checkHttpd.php output lighttpd output, killall -9 lighttpd to ensure shutdown, http fetch mute removed so error gets logged
  • update.php: removed apt updates because soft.sh already does this
  • crons: increased check http to 4 minute intervals (avoid http timeout period issues)

18/5/2011

Temporary fix to bypass userDb until fully utilized. You may use this update or use /scripts/util/populateUserDb.php to fix the issue.

  • Changed listUsers.php to always use systemUsers.php, as fixing userDb saving introduced a bug where ie. checkInstances was not working fully (using userDb, which lacks most users in most servers)

16/5/2011

Few bug fixes and minor features introduced.

  • Update script order of items.
  • Update script apt-get options for additional package installation lacked auto confirm.
  • Update script support for "step 2", /scripts/util/update-step2.php is included if present, thus supporting dynamic updates
  • Update script execute binary line added, so it can actually be executed simply by ./update.php
  • /scripts permission fix, only root should have anykind of access (no execute or read for others)
  • Support for root additional items via /etc/seedbox/config/root.cron.additional
  • Add user script minor cleanup, additional commenting.
  • Suspend script comments
  • simple terminate user script, ALPHA -> not feature complete yet.

Updating from 27/03/2011 release or earlier 

cd ~; bash soft.sh; php /scripts/update.php

Updating from 15/5/2011 release 

php /scripts/update.php


15/5/2011

Couple important new features introduced, several bug fixes.

  • Added lighttpd status checking and auto-restart (very rudimentary)
  • Fixed typo in /scripts/util/userConfig.php
  • Added /etc/seedbox/config/root.cron
  • /scripts/util/quotaFix.php now echoes results (also turns quota off, no need to turn it off first)
  • proftpd config update (remote) with MaxClientsPerUser, MaxClientsPerHost, run /scripts/util/ftpConfig.php
  • users class now supresses error if runtime db is not found
  • users class now saves the user db upon save
  • users class deconstructor added to save users db upon exit
  • modules support code begun (version for highly dynamic with multiple hooks etc.)
  • Changed update.php to a very basic one, as the remote capabilities are not being employed currently
  • Rudimentary module /etc/seedbox/module/basic/addUser.php for ultra simplistic user addition module (ie. custom configs, limitations).
  • Basic suspend & unsuspend capabilities (untested ALPHA)
  • skel permissions fixed

updating 

bash soft.sh; php /scripts/update.php


27/03/2011

This is primarily a bug fix and code quality release.

  • Set shell to /bin/bash upon user creation
  • Set shell to /bin/bash when using userConfig
  • quotaFix.php now turns quota off before attempting quota check
  • quotaFix.php now displays quota info before and after check
  • Lots of comments added to all code files touched
  • userConfig.php $filesLimitPerGb was undefined - hardcoded 500
  • update.php minor changes
  • default version now set to 1, in preparation to take it into production
  • startRtorrent: codestyle fixes, Made error messages more clear, named rTorrent screen session to "rtorrent"
Retrieved from "https://wiki.pulsedmedia.com/index.php?title=PM_Software_Stack&oldid=2064"