unsanity.org: love tropicana: the fix for securityd eatings gobs of ram when updating keychain entries

unsanity.org: love tropicana: the fix for securityd eatings gobs of ram when updating keychain entries home products company support contact weblog store « macworld 2007! | main | ape and rosetta: the battle's done and we kind of won so we sound our victory cheer » january 22, 2007 love tropicana: the fix for securityd eatings gobs of ram when updating keychain entries a few nights ago i was updating some not-to-be-named software on my laptop. this piece of software had a few passwords stored in the keychain. since said application was recently updated and therefore the code was modified, the system asked me if i wanted to give access to the keychain to this updated application. the dialog that it shown to the user is shown below: bad things happened when i clicked the "change all" button to once again allow this updated application to access all the passwords it was allowed to access. specifically, the securityd process was using 1.3-1.7gbs of ram (the rprvt value is all that matters). this was really, really bad as it caused my machine to page-out and page-in like crazy. due to the high memory usage, it also caused my boot volume to run out of space because of all of the swap files in /var/vm/. my point is that very, very, very bad things happened. after i cleared a lot of unused crap (garage band loops and old idvd themes) off my boot volume, i rebooted. i then tried launching the updated application again. i got the same dialog and the same problem. however, since i now had enough hard drive space available, i just waited for about 10 minutes. the passwords were accessed successfully. i then relaunched the application and securityd crashed. lovely. rebooting just repeated the cycle. also lovely. the securityd daemon handles all authorization and marshals keychain access. if securityd is not running, you cannot authorize any application (although sudo still functions) and all calls from applications to the keychain either immediately return an error without asking for a password or permission or they just freeze indefinitely. this is also very, very bad. it amounts to a denial of service since you're unable to do much with your computer. so i clearly had a problem. i remember seeing this problem before while browsing the internets. i specifically found it on 43 folders. i started searching the google for any solutions. the searches brought up some really amusing "solutions" such as a thread from mac os x hints. people are talking about some crazy voodoo solutions there. examples include updating the firmware or adding a "keychain" extension to the user's keychain. some voodoo causes are also mentioned. one such example is that it is cause by syncing keychains between an icbm and a powerpc-based mac. the troubleshooting ok, so obviously there were no known solutions to the problem that i could find. this kind of pissed me off. especially since i had to wake up at 5am to make it to a microsoft vista launch event. it was 3am at the time. obviously, there wasn't going to be any sleep for me. i decided to do my own troubleshooting. the crash of securityd clearly pointed to a corrupted file of some sort. so i had to determine which file was hosed like a bear coming into contact with the strikingly handsome stephen colbert. first, i restored all my keychains from known-good backups that another mac was actively using. i rebooted and tried to reproduce the problem. bloody 'ell! the problem remained. so it obviously wasn't that the keychains were corrupted. i rebooted again, started up the terminal, ran sudo fs_usage -f filesys securityd to see what files securityd was accessing when it was trying to update the keychain reference for the application. /var/db/systemkey repeatedly showed up. however, i was very, very tired and getting very, very angry so i decided to go with a machete approach to fix the problem. i ran strings on securityd and it turned up four "interesting" files and folders: /var/db/systemkey /var/db/tokencache (this is a folder). /var/db/systementropycache /var/db/codeequivalencedatabase since i was so angry, i just "moved aside" three of these. "moved aside" meaning that i just renamed everyone of them to have a ".old" extension using the terminal (via mv). the tokencache folder did not exist on my icbm so i couldn't move it aside. i rebooted. the problem was fixed!. i then went to the microsoft event. i emailed the very sexy merlin mann with the possible shoot-everything fix. he reported back that it fixed the problem for him, at least. hallelujah! the actual fix i was pissed off at the fact i wasn't able to narrow down the problem to one file. so after a very good 14 hour sleep (after being awake for over 30 hours), i set about to confirm the issue. i started by examining the files and comparing them to the files on my powerpc mac. the tokencache folder didn't exist on the icbm and the systemkey didn't exist on the powerpc mac. so i could rule those two out. i examined the systementrophycache file and its contents were boring. so i was able to rule that out. i then opened the codeequivalencedatabase in textmate (an awesomely awesome overnight text editor) and the text strings matched the applications that currently have keychain access. so i put the "moved aside" codeequivalencedatabase file back to its original location and rebooted. i then opened transmit (an awesomely awesome ftp application) which i had just updated and the problem reoccurred. yay! we (the royal we) had just narrowed down the exact corrupted file that was creating the problem. in order to fix this problem if you are having it, just open the terminal (/applications/utilities/terminal) and type: sudo mv /var/db/codeequivalencedatabase /var/db/codeequivalencedatabase.old or open /var/db (and then manually move codeequivalencedatabase to the trash, if you can). upon rebooting, god should be in his heaven and all should be well with the keychain. note 1: if codeequivalencedatabase is corrupted, then updating mac os x will also cause securityd to eat gobs of memory near the very end of the update cycle when the installer updates apple applications. this may make it seem like the update stalled. note 2: the /var/db folder is interesting. it's not required to boot mac os x and all the files in there are created as needed by mac os x. it's also explicitly ignored by repairing permissions. however, it does hold important account information, so i would not delete the folder or the items inside unless you are trying to revert your mac to "factory fresh" and do not wish to have any of the same accounts available without recreating them. with love, the malicious retard. posted by rosyna at january 22, 2007 12:09 am trackback pings: trackback url for this entry: http://www.unsanity.org/mt-tb.cgi/413. listed below are links to weblogs that reference love tropicana: the fix for securityd eatings gobs of ram when updating keychain entries: the fix for securityd eatings gobs of ram when updating keychain entries from [linked]http://www.unsanity.org/archives/security/love_tropicana.php[read more] tracked on january 22, 2007 12:30 pm related: leopard! - oct 27, 2007 smart crash reports: 300! - jul 16, 2007 fontcard 1.5b2: fap, fex, and suitcase fusion - may 24, 2007 my dmg is bwoken after download! - may 17, 2007 label my recent items, ye olde skool style - apr 20, 2007 comments very interesting -- i love it when the root cause is eventually found. would you mind posting a link to the application mentioned in the screenshot? ;) posted by: stephen on january 22, 2007 6:46 am moving /var/db/ files seems like a bad idea out of principle. is there any easy way to figure out if the codeequivalencedatabase file is used by anything else? does the file regenerate? etc. but on the other hand, this thing is filled with pretty useless, empty data...meh. posted by: steve streza on january 22, 2007 11:07 am yes, codeequivalencedatabase, like all files in /var/db/, codeequivalencedatabase is regenerated when needed. and yes, codeequivalencedatabase is used by the keychain for hashing applications only. posted by: rosyna on january 22, 2007 11:09 am big, big thank you for this troubleshooting. this tiger vm exhaustion problem has been afflicting my powerbook for weeks, maybe months, repeatedly causing seize-up at just the wrong moment. _and_ done raspberries at me for not being able to troubleshoot down to its cause. you da (wo)?man! posted by: mike on january 27, 2007 3:53 pm unsanity, the world has changed -- there is an urgent need to name those that deserve to be treated as person should be, thou whose used to be "things" or "it" (meaning of my point is entities of type "application", "tool", "intricate part of the operating system"). we think you must discover-construct new kind-of haxie having an ability that is required in the new times you observe all around, unsanity. it's the capability to name and then [follows] ability to call a thing "person". to do so, every application system attaches to 'mac os x gui' interface (indirectly) should have smart attribute complex that gives your users a way to give a *unique*, generated by human ingenuity and fitting form and function {name-of-self}. therewore it will be possible for people to address toward those named in the same manner as you command persona with a given name, unsanity. what follows, users will pay more attention to such "unimportant details" as messages presented to them in case there's a need to warn about possible negative side-effects people /usually/ ignore because of everyday routine. unsanity, consider our suggestion - think of the future of the web and consider the present. i shouldn't tell you that, but if you brainstorm about { this } pick a very love price even if it takes more than three days of coding in to make one. do test it. twice. naming is a kind of magic :-). it does simply work, unsanity. it does, believe me, it does simply work (...not on women only :-)). i have all your haxies tested on all machines (except the critical ones) and this few silly ones (...like paranoid android) are beautifuly made. good work, you know your craft, unsanity. take care, dr. miranda mckennitt-knott p.s. do not ignore this note - what was said is very important in not so distant future! posted by: dr. miranda on january 29, 2007 6:31 am so, where can i grab that app? ;-) posted by: phil holland on february 4, 2007 1:55 pm where can i get a copy of pornography viewer 6.9? posted by: vector on february 4, 2007 3:03 pm i totally didn't get a chane to talk to you at the mda/mh party. what a shame. anyways, your cocoa duel website is being nastily spammed. just a heads up. posted by: ernest liu on february 4, 2007 3:18 pm thanks for posting this, saved me the massive troubleshooting effort you had to endure. posted by: hans on march 15, 2007 7:41 am thanks! this saved me from throwing my imac out of the window after an archive & install + update to 10.4.9 started giving me the exact same problem. one quick rm of codeequivalencedatabase and a reboot later, and all was back to happiness! posted by: matt on march 17, 2007 11:11 am absolutely brilliant! i stumbled upon this article a couple of months ago and now i am experiencing this exact problem! thanks for going through the motions of tracking down the culprit! posted by: joakim nygård on march 31, 2007 5:18 am nice troubleshooting story. however, i just want to point out something possibly dangerous: like all files in /var/db/ ... regenerated when needed isn't /var/db/netinfo/local.nidb/ the master copy of the local netinfo database? i'd hate to see somebody blow that directory away, assuming it would be regenerated. posted by: matt on april 22, 2007 1:02 am matt, yes. however, if you blow it away, it will be recreated. you'll just uhm... possibly lose all access to your accounts. which is what i meant by the next sentence, "however, it does hold important account information" posted by: rosyna on april 22, 2007 1:05 am hi! friday i was updated with the last security update and when i restart start having this problem (securityd) i try to run the command in terminal but when it ask me for password i type it and says me "sorry, try again" what can i do?? i typing the right password but it donts accept it. please help!! leo. posted by: leo on april 23, 2007 7:18 am hello again! everything works fine now! apparently whe i was trying to run the command "securityd process" was killed for me thats why terminal doesnt allow me to run the command. i reboot and run the command without problems, boot again and the problem was fixed!!! thanks!! posted by: leo on april 23, 2007 7:39 am ha! my pv is at 5.8.1 posted by: bug on may 3, 2007 3:27 pm many thanks as you said me a lot of aggravation and time. this problem has plagued my machine for about a month and i recently noticed the memory usage of securityd leading my google search here. if you're ever in the allentown, pa area the beers are on me! posted by: mark on june 16, 2007 6:26 am as far as i know the /var folder contains "variable" files, files that change during normal operation. i think that the rule was that application should not expect that these files to be present after reboot and should therefor be able to recreate them when needed. ref: http://en.wikipedia.org/wiki/filesystem_hierarchy_standard posted by: gmlk on june 22, 2007 11:30 am warning - be careful with codeequivalence and the /var/db/ contents. not all files are recreated by reboot. i deleted the codeequivalencedatabase _and_ the codeequivalencecandidates files and now the mac in question (thankfully not my day-to-day system) is broken in a big way - keychains are not working at all and can't be repaired with keychain access. the codeequivalencecandidates file is not recreated with anything i do. i have replaced it with a known good and still the system is as if it knows nothing about keychains. i created a new admin level user and the ~/library for the new user has no keychains folder!!! do not delete the codeequivalencecandidates file under any circumstances. posted by: peter b on july 3, 2007 6:29 pm warning update - the mac i broke as described above was as a result of me trying to refresh it manually for a new use without doing a full system install. as well as deleting both the codeeq files as described, i also emptied the /private/var/tmp/ directory thinking that it was safe to do so. the combination of both of these changes seemed to stop keychains from working cleanly all together. it wasn't until i rebooted a couple of times, and the /private/var/tmp/ had a chance to rebuild its contents that keychains worked again. i have no idea what else is broken on this system from what i have done and will rebuild it from scratch. just hoping to avoid others making the same stupid assumptions i did. it seems obvious now that the os does not empty /private/var/tmp/ on reboot like it does the /private/tmp/ directory for very good reasons. posted by: peter b on july 3, 2007 7:09 pm thanks!!! it was driving me crazy. i'm really pissed at apple for messing up so many things during recent upgrades. posted by: gerald on august 13, 2007 4:47 am thank you, you saved my ass. the problem surged after the july security upgrade... posted by: discordian on august 14, 2007 6:32 am yet another huge thank you for figuring out the cause of this problem. big sigh of relief. you rock! posted by: bruce on august 24, 2007 2:42 pm thanks for this solution. you're great. posted by: mistamilla on august 28, 2007 1:04 pm the immense amount of thanks i wish to shower upon you for blogging this cannot be described. now i wonder whether this "just happens" or if it only happens to geeks who mess around where they shouldn't on their macs. posted by: brent_s on august 31, 2007 8:28 am huge thank you! i'd been frustrating these couple of months. apple should add this information their support documents... posted by: ushi on september 8, 2007 2:46 am works here. huge fix. thanks. posted by: double bee on september 27, 2007 10:24 pm saved my ass. this will be featured on my podcast tonight www.typicalmacuser.com as this thing hit me hard last week and this fixed most of it. you guys rock! posted by: victor on october 9, 2007 10:56 am just to chime in... i'd been having freeze problems with my macbook pro for months so much so that i dreaded seeing the keychain update dialog! awesome fix! posted by: ken_ray on november 3, 2007 1:26 pm post a comment name: email address: url: remember me? yesno comments: (you may use html tags for style)

Acceuil

suivante

unsanity.org: love tropicana: the fix for securityd eatings gobs of ram when updating keychain entries Forum CAC 40 Volumes anormaux au fix de vendredi sur les valeurs ... FORTIS B FIX S.A. Sicav de droit belge Montagne du Parc, 3 1000 ... FORTIS B FIX S.A. Sicav de droit belge Montagne du Parc, 3 1000 ... FiX-Netze.com � Blog Archive � Un MacOS X pas comme les autres� El-Fix a/s Bored fix - Why should you be bored? FIX sugar fix recordings The Fix Salon Arts & Entertainment Celyatis, Incontinence, Slip filet Tena Fix, sante, confort et ... Couche lavable Easy Fix aLaide.com - Dictionnaire - D�finition : Code-and-fix Neo Fix XIV PSTwo - : FOXCHIP : Modification et R�paration des ... T�l�charger Fix My Registry 2.3 gratuite en fran�aise - Brothersoft.fr T�l�charger Dial-a-fix 0.57.7 Stable Full gratuite en fran�aise ... Acheter Couche lavable Easy fix de popolini... avec eco-SAPIENS Identity Theft Fraud Resource - Insurance & Repair Services ... Descriptif du produit Tol-Fix de Tollens sur Batiproduits.com Shop.WND.com - A WorldNetDaily Exclusive! Thermal Fix Serum de Vichy Chanceli�re Cabrio Fix Number One - Accessoires poussette ... 飞客数据恢复中心�硬盘RAID数据恢复{北京\上海�}电话：800-810-6696 Carrossiers ind�pendant au Canada - Fix Auto Carrossier Logic-Immo : Immobilier Fix-saint-geneys 予定調整ツール：fix on Emploi FIX Technical Sales (OMS, FIX, Algorithmic Trading Sales ... Fix ATI pour Counter-Strike 1.3 - PC INpact Winamp Updated to Fix Security Hole Le blog du groupe ID-FIX - Espace Perso HitMuse de ID-FIX Bicycle Repair and Maintenance: Bicycling Magazine.com IONA Financial Solutions: FIX Information Chanceli�re Cabrio Fix Athletic : enfant - article b�b� � petit prix Couverture Zewi-Fix Ciel 90 x 190/200 cm Zewi B�b� jou (Zewi B�b� jou) Couverture Zewi-Fix Bleu 60 x 120 cm Zewi B�b� jou (Zewi B�b� jou) Daily Funny Fix / Media T�l�charger Norton 2000 BIOS Test/Fix: version 1.0 [Freeware ... BUG FIX d�finition BUG FIX CRAOWIKI - Fix Fix T�l�charger Div Divx Fix Repair Joiner. Fixez le dossier endommag� ... The Daily Fix JScreenFix - Fix stuck pixels Fix My Essay: Personal Statement and Admissions Essay Help ... Why Search Sucks & You Won't Fix It The Way You Think Fix It Tools - cheap power tools, discount air tools and hand tools Fix-max, fix-wear, fixmax, se fixe partout! XTREM'FIX - produits professionnels Bostik PocketPCFreeware : Notification Clear Fix 1.2 Ultim'Fix Spray Coiffant Studio Line de L'Or�al Paris East Bay SPCA Porte-bidon, bottle fix kit de fixation pour porte bidon engrais - Bio Fix Grotek McKenzie can fix front-row woes - Rugby - Fox Sports Permanent Fix for the Shmoo Group exploit - The OLD TechLifeBlogged DIVFIX.MAXELINE.COM - Offical DivFix homepage - divx, avi, video ... Firefox Bug 246078 Fix :: Mozilla Stuff :: JohnHaller.com HOUSE OF FIX - LIMITED EDITION T-SHIRTS ! - Les news de Bikini Test WinSock XP Fix 1.2 D�tails constructifs. CYPE. FIX: Planchers Inclin�s. D�tails ... tono fix you (Tono Monof�nico fix you) (sonnerie.01net.com) Quick-Fix Keychain