- Make scripts_plugin email heuristic less stupid, or maybe even ask for an email. This is tracked as Scripts #224 (this issue) and Scripts #193 (tracking a contact address). - Current parallelization probably does a bad job distributing working tasks over different components of the pipeline. Fix this by adding jitter? Trying to smear things out? - Test head doesn't do quite the right thing with version numbers (shouldn't git describe, instead should give a version infinitely in the future.) - Strategy introspection and disabling. - prepare-config (and others) create .wizard dir even when not strictly necessary - Bug out immediately if tags are not present in the master tip of the repository - pending doesn't seem to get written out properly sometimes (or it's being deleted); this makes it hard to --continue on the event of an upgrade failure. Also, we seem to bounce back to the production copy to check pending even when we run --continue from the working dir. - Newline checks are /really really/ expensive on AFS; see if we can minimize them or something. Right now, we're testing a fix where we don't clone with --shared. - Replace .split("\n") with .splitlines() - Need to fix existing repo history? (not adding extra commits; that'll be more difficult) git rebase -i -p --root --onto COMMITID This won't work if you need to change the very root of the repository. You'll probably end up with conflicts and have to manually resolve everything afterwards. But usually you won't need --root --onto unless you really fucked up the pristine branch. If you just need to change the scripts spine, git rebase -i -p COMMITID should work. - [SCRIPTS] MediaWiki 1.6.7, 1.9.3 and 1.10.0 - geofft comments: "Connection to scripts.mit.edu closed" is confusing (tracked #393) the URL should be easier to copy and paste, which means we should move it out of dialog We should ... upgrade our autoinstaller Apparently installing WordPress updates or themes never indicates completion, and just says "Downloading..", and you have to guess when it's done - [SCRIPTS] phpBB - phpBB or phpbb? (right now it's the former) - need an upgrade story; srv needs more versions - need a story about install/ contrib/ - Give users a "certificate" of their merge, which they can use to reuse that merge commit if something unrelated fails. - Human readable quota output - Nice error message on --continue if you forgot to git add your resolved file (look for conflict markers) - The merge interface is a kind of major UI disaster; you won't be able to use it unless you know how Git works. Also, the merges can be quite difficult to resolve if upstream has made large formatting changes like reindenting. We may also consider providing a --rebase option, which seems to do better when big problems like this show up. - Better mass-* support for just one user (this includes letting a user mass upgrade just his own autoinstalls) - Show progress or something when upgrading - Allow 'sticky notes' for future upgraders to notice - .wizard/url semantics are subtly wrong: in particular, if we explicitly configure a URL at install, we should be able to detect this URL as baked in from the configuration - Rerere support doesn't actually work - "Version 3.0.0 doesn't exist; did you mean 3.0?" - Be a little more intelligent when perform web checks; for example, if we get a forbidden message, that probably means we go the right address but it's blocked off; if we get a 404 message, that probably means wrong address. Account Unknown is something particularly good to check for. - Wordpress module can do something intelligent if we get redirected to the installation page. - wizardResolve* files seem to get left in tmp en-mass, and we don't know why. - [SCRIPTS] Wordpress needs to get rid of the siteurl hack, so that it actually has a fully-qualified URL http://foo.scripts.mit.edu/blah. This will also fix Wordpress's cron functionality. We should be careful not to write over users who are on vhosts. We should figure out who is still on twiddle paths. We should make sure the redirect is handled correctly. - Remerges aren't reflected in the parent files, so `git diff` output is spurious. Not sure how to fix this w/o tree hackery. - Sometimes users remove files. Well, if those files change, they automatically get marked as conflicted. Maybe we should say for certain files "if they're gone, they're gone forever"? What is the proper resolution? - Parse output HTML for class="error" and give those errors back to the user (done), then boot them back into configure so they can enter in something different - [SCRIPTS] If you try to do an install on scripts w/o sql, it will sign you up but fail to write the sql.cnf file. This sucks. - [SCRIPTS] Web application for installing autoinstalls has a hard problem with credentials (as well as installations that are not conducted on an Athena machine.) We have some crazy ideas involving a signed Java applet that uses jsch to SSH into athena.dialup and perform operations. - Pay back code debt - Tidy up common code in callAsUser and drop_priviledges in shell; namely cooking up the sudo and environment variable lines - Summary script should be more machine friendly, and should not output summary charts when I increase specificity - util.fetch() should use urllib under the hood, not httplib. Code has to be changed. We should log if we get redirected. - Other stuff - Add support for mypristine workflow - Figure out why Sphinx sometimes fails to crossref :func: but wil crossref :meth:, even though the dest is very clearly a function. Example: :func:`wizard.app.php.re_var` - The TODO extension for Sphinx doesn't properly force a full-rebuild - Make single user mass-migrate work when not logged in as root. The primary difficulty is making the parallel-find information easily accessible to individual users: perhaps we can do a single-user parallel-find on the fly. - Don't use the scripts heuristics unless we're on scripts with the AFS patch. Check with `fs sysname` - Make 'wizard summary' generate nice pretty graphs of installs by date (more histograms, will need to check actual .scripts-version files.) - It should be able to handle installs like Django where there's a component that gets installed in web_scripts and another directory that gets installed in Scripts. - [SCRIPTS] ACLs is a starting point for sending mail to users, but it has several failure modes: - Old maintainers who don't care who are still on the ACL - Private AFS groups that aren't mailing lists and that we can't get to A question is whether or not sending mail actually helps us: many users will probably have to come back to us for help; many other users won't care. - Whatever happens here should be used to improve user.email() [ XXX: metadata.rst ] COMMIT MESSAGE FIELDS: Installed-by: username@hostname Pre-commit-by: Real Name Upgraded-by: Real Name Migrated-by: Real Name Wizard-revision: abcdef1234567890 Wizard-args: /wizard/bin/wizard foo bar baz GIT COMMIT FIELDS: Committer: Real Name Author: lockername locker OVERALL PLAN: [ XXX: doc/deps.rst ] * Some parts of the infrastructure will not be touched, although I plan on documenting them. Specifically, we will be keeping: - get-homedirs.sh. which needs to be run as root on scripts. Store in /mit/scripts/sec-tools/store/scriptslist - parallel-find.pl, and the resulting /mit/scripts/sec-tools/store/versions [ XXX: doc/deploy.rst ] GET APPROVAL BEFORE PROCEEDING ANY FURTHER; THIS IS PUSHING THE CHANGES TO THE PUBLIC NOTE: The following commands are to be run on not-backward.mit.edu. You'll need to add daemon.scripts-security-upd to scripts-security-upd to get bits to do this. Make sure you remove these bits when you're done. 10. Run `wizard research appname` which uses Git commands to check how many working copies apply the change cleanly, and writes out a logfile with the working copies that don't apply cleanly. It also tells us about "corrupt" working copies, i.e. working copies that have over a certain threshold of changes. 11. Run `wizard mass-upgrade appname`, which applies the update to all working copies possible. 12. Run parallel-find.pl to update our inventory [ XXX: doc/metadata.rst ] * The repository for a given application will contain the following files: - The actual application's files, as from the official tarball - A .scripts directory, with the intent of holding Scripts specific files if they become necessary. - .scripts/dsn, overriding database source name