Dataloss Bug in Latest Windows Installer Builds Can Delete Contents of Program Files FolderThursday February 5th, 2004A serious bug in the latest Windows installer builds of the Mozilla Application Suite can lead to users having the entire contents of their Program Files folder deleted. The problem is present in the 2004-02-03-18 and later trunk Windows installer and net installer builds. The affected binaries have had the suffix -dangerous added to their filenames and been removed from the latest-trunk and latest directories on the mozilla.org FTP site. Testers are strongly advised not to download these dangerous builds. The issue is being tracked in bug 233014 (please do not add unnecessary comments to this bug). Thanks to Zbigniew Braniecki for alerting us to this problem. Update: The bug is now fixed and latest Windows installer and net installer builds do not exhibit the problem. Is this similar to the Mozilla Firebird Install bug ("safe install")? At all - yes. This bug is about different problem, but making installer delete only self-created files (and extensions/skins) would make this bug not important Why oh why is the installer deleting things anyway? A convenience to people using the nightlies? This is the job of the uninstaller - it should be invoked if you to clean up first. The problem is with extensions, which are sometimes incompatible with newer versions. If you simply do a surgical removal of official Mozilla files (which the uninstaller does), the extensions are left behind. If you then install a new Mozilla version, it may well crash because of the old extensions. There's no way of surgically removing old extensions - they're all built differently and the installer has no way of knowing which files are which. The only way to be sure of getting rid of them is to nuke the whole folder. Why do extensions have to be deleted anyway? Any new browsers should load only "certified" extensions, or prompt for extensions upgrade upon installation. If you choose not to upgrade extensions, they won't be loaded. If you do choose to upgrade, each mozdev subsite site will check that the version is compatible and "certify" it. If it is incompatible, prompt for a new extension istallation. And why isn't there a method for uninstalling extensions? That seems like quite an over sight. I believe this is because extensions can do far more than just dump a jar file in your profile directory and add a few lines to installed-chrome.txt. I believe, for example, that it is quite possible for an extension to write files to anywhere on the disc to which the user has access. Given this, it's prett hard to write an uninstaller that works in all cases without ever causing dataloss. There has certianly been talk of writing an uninstaller tpo deal with the common case of chrome directory installations. Of course, talk is useless unless it's backed up by code. A Sanity check for the mozilla .exe would stop dataloss in most 'sane' systems. It's going to look like that "most" isn't good enough... No doubt that the Mozilla.org team is discussing the functionality of the installer internally. And the resulting quality is going to match that of the browsers. Perhaps the "delete all files" method of installation should only delete KNOWN directories from the base install directory and leave any others alone -- that is: bin, chrome, components, chrome, components, defaults, Plugins, res, searchplugins, uninstall. Any non-directory files at the base install directory would be deleted, but directories not in the above list would be left alone. This would allow for complete uninstallation of plugins, etc. but keep any user-created directories. So in most cases, installing to C:\Program Files on Windows (rather than e.g. C:\Program Files\Mozilla) would do no damage, as most (all?) programs use their own folders. Good idea. No, you have it backwards. Installing to c:\Program Files IS the problem. That is what this whole bug is about. Somehow the default install location became "C:\Program Files" instead of "C:\Program Files\mozilla.org" as it traditionally has been. Someone want to note that the "latest" windows installer doesn't have this problem and that the builds were already fixed by the time this story went to press? You sound like you think that people should have known it was fixed. Leaf's comment on the bug was "I've reverted the path on the build system, so future builds will not have this issue, but it would be good to bulletproof the installer creation system against line ending issues caused by using cygwin perl/utils", and the bug wasn't marked fixed. Hardly a clear announcement. Morphing that bug was a dumb idea - given that end-users are taking their information from it, it should have been marked FIXED with a comment that was clear to non-techies, and a new bug filed for the bulletproofing fix up. Alternatively, one of you mozilla.org folks could have posted about the problem, and the fix, in a newsgroup (or submitted a story here, or used your blog), so that the communication wasn't relying on the bug and random people posting stuff in various places. "Morphing that bug was a dumb idea - given that end-users are taking their information from it." Bugzilla is not an end-user database and nightly builds are not end-user products. End users shouldn't be taking anything from Bugzilla and shouldn't care what goes on there. The builds that were broken were labeled as broken and mozillazine ran a story noting the broken builds. I submitted a request to mozillazine that they update the story noting that the fix had gone in before the story ran. That update has been added to the story. Nightly builds _MAY_FORMAT_YOUR_HARD_DRIVE_AND_MELT_YOUR_COMPUTER_ They'll probably kill some babies too. Use at your own risk. So do you recommend against testing Mozilla nightly builds except on a separate computer used solely for this purpose? I recommend testing Mozilla however you feel comfortable testing it. I test every day on my personal and work computers. I also back up my most important data regularly and I know that any day Mozilla could wipe it. Truth is, however, that I've had more massive and critical failures from my hard drives going bad (three in three years) than I've had from Mozilla nightly builds (only one major dataloss in 4 years). Backing up data is just good practice, especially if you're using a lot of in-development software. I just read leaf's comment again and it doesn't look that ambiguous to me. If you're "involved" enough to be downloading builds and reading bugs, then you should be able to parse "I've reverted the path on the build system, so future builds will not have this issue", look at the time the comment was made, and check ftp to see if there are builds newer than that. It's really not unclear, to anyone that belongs in Bugzilla in the first place. I didn't say it was ambiguous, I said it wasn't a clear announcement. It was fine for anyone that "belongs in bugzilla", but there are/were lots of people reading it that probably don't belong in bugzilla, and shouldn't be using nightly builds. It's quite obvious from reading forum discussions that there are people using nightly builds that don't know how to use bugzilla, and don't understand concepts of trunk and branch, or the even the difference between a nightly and a alpha/beta. I understand that this isn't supposed to be the case, but it is. Aside from that, there are people that don't actually use nightlies, but pick up on issues from forums (and in this case Mozillazine stories). You can either acknowledge that that's the reality and work with it, or you can live with newsgroup/forum posts saying "OMG WTF MOZILA ATE MY HARD DRIVE!!!1!! THIS SUXORS!!! IM GOING BACK TO IE". If you're aiming at end-users, you either need to cater to end-users with bugzilla comments and nightlies, or you need to keep them away from them. Currently we have "end-users" following the day-to-day development stuff, and then becoming upset when it doesn't do what they want (or eats their program files). "It's quite obvious from reading forum discussions that there are people using nightly builds that don't know how to use bugzilla, and don't understand concepts of trunk and branch, or the even the difference between a nightly and a alpha/beta." "We make binary versions of Mozilla available for testing purposes only. We write code and post the results right away so people like you can join our testing process, try it out, and report the bugs you find. It's guaranteed that you'll find bugs. Lots of them. Mozilla might crash on startup. It might delete all your files and cause your computer to burst into flames. Don't bother downloading Mozilla if you're unwilling to put up with problems." I don't have time to worry about people who can't read warnings or read them and don't heed them. I used to have time to care. I used to put up notes nearly every single day about the quality, stability and features of each nightly build. I'd test within minutes of the builds being made available and post my results as soon as my smoketesting was done. I did this literally for years. I don't have the time to do that any more. If ths community in the forums wants to help protect its participants, maybe some champions there would like to devote that time to testing and putting up a daily status each morning after the builds are posted. When I was decreasing the frequency of my daily build comments, I seem to remember the forums, and Jason Bassford in pariticular, picking up the slack and there being regular daily build comment threads and status. Has that stopped? If so, too bad. Maybe someone will pick that effort back up. It's not gonna be me. "If you're aiming at end-users, you either need to cater to end-users with bugzilla comments and nightlies, or you need to keep them away from them." I'm certainly not encouraging end-users to use nightly builds and participate in Bugzilla. At the same time, I'm not about to password protect our nightly build directory and ask that all Bugzilla participants have degrees in software engineering. "Currently we have "end-users" following the day-to-day development stuff, and then becoming upset when it doesn't do what they want (or eats their program files)." I imagine they'll stop after a couple of melted hard drives :-) perhaps we could call this the "switch to linux, resistance is futile" bug....
Or the "windows users avoid Mozilla if you like your files" bug..... Do we need a system to handle emergencies faster? The bug was fixed quickly (thanks to whomever patched it), but I'm concerned about containing damage: According to the bug, and perhaps something was done but not reported in the bug, it took 17 hours to contain the damage. That's 17 hours of people downloading builds and suffering dataloss. bug reported: 2004-02-03 19:52 PST action taken to contain damage: 2004-02-04 12:50 PST: http://bugzilla.mozilla.org/show_bug.cgi?id=233014#c26 I renamed the builds in 2004-02-03-18-trunk and 2004-02-04-12-trunk by adding -dangerous to the end of the filename, and removed them from latest/latest-trunk.
Wow...personally I think you should be impressed with the speediness of the action. 17 hours is nothing! Remember all these downloads you're talking about would be nightlies...nowhere does anybody suggest installing nightlies is a safe thing to do. In fact pages on moz.org even warn you about the dangers. Let's not overreact "We make binary versions of Mozilla available for testing purposes only. We write code and post the results right away so people like you can join our testing process, try it out, and report the bugs you find. It's guaranteed that you'll find bugs. Lots of them. Mozilla might crash on startup. It might delete all your files and cause your computer to burst into flames. Don't bother downloading Mozilla if you're unwilling to put up with problems." Of particular note are the last two sentences there: "It might delete all your files and cause your computer to burst into flames. Don't bother downloading Mozilla if you're unwilling to put up with problems." http://www.mozilla.org/binaries.html Except the guidelines also say don't report bugs against the latest milestone build. Most people who find a reproducible bug in a milestone build don't have enough money to buy a dedicated computer just for installing the latest nightly so that he or she can report it in Bugzilla. "Except the guidelines also say don't report bugs against the latest milestone build. Most people who find a reproducible bug in a milestone build don't have enough money to buy a dedicated computer just for installing the latest nightly so that he or she can report it in Bugzilla." Then I guess we'll have to do without that bug report. It's not like we've got a major shortage of bug reporters and reports. If a potential bug reporter isn't comfortable taking the risk of using nightly builds then he can just not report it. Look, we've had maybe 3 or 4 crirical dataloss bugs, almost all of them Mozilla data, in the 5+ years that I've been using nightly builds. We've been cranking out two rounds of builds a day for several of those years. By my math it works out to between 1/10th and 2/10ths of 1% of our builds that have had major problems. If you or anyone else isn't comfortable with that risk then don't download a nightly build. No one's holding a gun to your head. Asa - that binaries.html page is fine. Unfortunately, it's not actually linked to from anywhere - http://www.google.com/search?q=link%3Ahttp%3A%2F%2Fwww.mozilla.org%2Fbinaries.html (well actually it's linked from a netscape.com page about Netscape 6 email, from Mozillaquest, and from mozilla.org's products.html which is also not linked from anywhere, but it's not linked from any current pages on mozilla.org) well, the warning used to be more visible. fixed. Well that's one spot fixed, which is good. Thanks. However, it's still possible to get the nightly builds straight from the sidebar on various pages (developer pages, and also /releases/ which is linked from obvious places including the homepage). The "nightly builds" sidebar link should really point to /binaries.html instead of going straight to ftp.mozilla.org. There's a bug filed about fixing that, but few people have access to change the navigation stuff (i.e. it'd be nice if you could just fix it like you did with the other page :) Actually, mozillazine already has a mechanism for posting a big warning box up at the top of the page when something really bad creeps into the build. Back when I used to do the BuildBar and daily build updates, I used it to post warnings for particulary scary builds. Mozillazine probably still has that big red box but they need to hear about problems in order to use it and they need accurate information about what's broken and when it's fixed. I think many people have gotten used to Mozilla nightly builds being fairly stable. We all need to remember that the nightly builds are all works in progress, and may potentially crash, refuse to start, scramble bookmarks, or even delete your entire hard drive. People who test these builds are taking a risk, but that risk pays off in the form of well tested stable release builds. Please read the warning at the top of this page: http://www.mozilla.org/binaries.html At no point has it been advocated that running a nightly is a "safe" thing to do. All the linux books I have read shout and point out: **Never use the system as root unless you want to change something important to the system** And administrator's account in windows 2K,XP is just as powerfull as the root account. Full and unrestricted access to all files and resources!!! More than enough for a poorly programed program (or even worse a malicious program) to destroy everything. And most users use their windows systems as administrators. That's in my opinion the worst vulnerability of windows systems!! I hope the paranoia will stop and people will stop to use the administrator account for everyday computing. And microsoft should definetely work towards this goal. So in my opinion what happened to many "volunteers testing mozilla" IS NOT a mozilla's fault! It's a problem with Windows configuration! I would like to hear your coments... All the linux books I have read shout and point out: **Never use the system as root unless you want to change something important to the system** And administrator's account in windows 2K,XP is just as powerfull as the root account. Full and unrestricted access to all files and resources!!! More than enough for a poorly programed program (or even worse a malicious program) to destroy everything. And most users use their windows systems as administrators. That's in my opinion the worst vulnerability of windows systems!! I hope the paranoia will stop and people will stop to use the administrator account for everyday computing. And microsoft should definetely work towards this goal. So in my opinion what happened to many "volunteers testing mozilla" IS NOT a mozilla's fault! It's a problem with Windows configuration! I would like to hear your coments... This is how I look at it: There was a simple flaw in the installer that caused this, not your whatever it is OS configuration. True, mozilla shouldn't remove files from directories outside its scope. Also, I do agree that way to many people use their root/administrator accounts for day to day use. This is wrong, but so is ... you are right , it's not mozillas fault. it's the fault of those window-users, damn bastards. *****sarcasme***** I never install any programs into the default windows OS partition on any of my computers, which is usually the "C" drive, other than included or added hardware. I add software and downloads into another partition which is usually the "D" drive. I also have an external hard drive that I use to test "beta apps" or programs that may be incompatible with the OS I am using. I also have a software program the monitors downloads, and or software installation, for easy uninstalls ( many add to or change registry settings). It also helps to create a restore point prior to adding any downloads or installing new software, with Windows XP. ( I don't know if this function is available with earlier windows OSes). Also, backing up data is important, as if a user is root or administrator, one slip can undo weeks, if not months of work. All in all, is it possible to have one stable build of mozilla located on one partition, and a nightly test build installed on another?? I haven't tried this, and wonder if it won't work , as I don't know what, if any, changes are made to the windows registry when Mozilla is first installed.
Only if you are not looking at the path were it should be installed. O.K., I was alert because I wanted to check this bug, otherwise maybe I would have overlooked that short path. Normally, I´m very careful about those things, but when you are doing all days the same, installing a nightly, sometimes you are just clicking, instead of looking first. herman Only XP-aware applications really work well under non-administrator accounts. what's worse xp allows you to have multiple administrator accounts so that Sam and Administrator (two seperate accounts) are both actually administrator. XP made me set it up that way during setup. Windows works better that way.. why? Because classic applications demand root access.. and could crash if not given it. They place their files in places a user could not access. How about not using installers for nightly builds? That way, only people who can unzip to a seperate, clean directory can get them. I never "install" a Firebird nightly. I've always known that the installer could contain additional bugs--write bugs that could mess up the operation of my computer or the integrity of my data. --Sam "How about not using installers for nightly builds? That way, only people who can unzip to a seperate, clean directory can get them." There are plenty of other parts of Mozilla that could cause major dataloss. This probably wouldn't make testers appreciably safer and it would mean that we'd be that much more likely to ship a dangerous installer in a milestone release that goes out to orders of magnitude more people. Doesn't seem like a good idea to me. |