Rails ActiveModel/ActiveRecord XML/JSON serialization replacement

I need a bit more control over the XML schema than the standard ActiveRecord/ActiveModel XML serializers provide, and a bit of control over root tags and attribute names in JSON that normally isn't there. I don't need the full control that something like RABL provides, and I'd like deserialization from XML to be automatically available for any XML I can serialize to. None of the existing gems provides what I want without having to hand-code methods on all classes involved, especially the deserialization part. And if I've written the deserialization code and have the hashes to control that, the serialization code based on those same hashes won't be that much additional work. The hard part will be eventually making it a drop-in replacement for the ActiveModel XML and JSON serializers and compatible with the parts of ActiveRecord that interact with serialization. I'll deal with that later, though, I'm just going to lay the groundwork for compatibility now and deal with actually integrating it once I've got the code working outside that framework.



XML serialization/deserialization description and control data:

Item hash:

• name: element or attribute name
• source: object attribute name, literal data string, nil for no contents
• +type: Ruby class name, Boolean short for "TrueClass or FalseClass", Literal for a literal element (no source attribute), default String if this is not the root element or the class being deserialized for the root element
• options: {options hash}, default {}
• *attributes: [array of item hashes describing attributes of this element], default []
• *children: [array of item hashes describing child elements of this element], default []

+ = needed only during deserialization

* = valid only in element item hashes, ignored in attribute item hashes

Options hash:

• *default: default value if source's value is nil, default no value
• trim: true/false, trim leading/trailing whitespace, default false
• elide: true/false, omit if value (after trimming if relevant) is nil or empty, default false

* = not applicable to items of Literal type or items where source is nil.

Control attributes on the object:

xml_attributes: an item hash describing the root element of the object

• Canonically the source would be nil, but a source attribute is legal if the root element will have actual text content.
• The type attribute of the root element defaults to the type of the object if not specified, and canonically it isn't specified since forcing a mismatching type will cause problems.
• The name attribute can be overridden by specifying an element name when serializing the object. When deserializing the root element's name is ignored and it's assumed the caller is deserializing the correct class.

attributes: a hash describing the attributes to be serialized

• The key is the source attribute name.
• The value is nil or a string giving the element name to be used.

If xml_attributes is not present, attributes will be used instead. One or the other must be provided.

When serializing XML:

• If attributes is used then the attributes are serialized as child elements of the root and the default for include_types is true and use_source_names is true if the value is nil and false if the value names an element name. If xml_attributes is used the default for include_types and use_source_names is false. This allows the XML to be unchanged if an old-style attributes hash is being used.

When serializing JSON:

• If xml_attributes is used then the attributes sub-hash is serialized first followed by the children sub-hash. Items with a nil source or Literal for type are ignored, they're relevant only to XML serialization.
• When deserializing, if type is absent then the type is determined by Ruby's default rules and the format of the value. Contained objects will end up deserialized to a hash and a class-specific attributes= method will be needed to recognize the attribute name, create an object of the correct class and initialize it with the hash before assigning it to the attribute.

to_xml options:

• root_name: name of the root element if not the class name of the object
• include_types: include type and nil attributes in XML
• use_source_names: follow Ruby's naming conventions for element names

root_name is normally used when attributes is used to control serialization, or if the root element name in xml_attributes needs to be overridden. Setting both include_types and use_source_names to true will yield the same XML normally produced by the old serializer when using xml_attributes .

to_json options:

• include_root_in_json: include the class name as the root element of the JSON representation, default false
• include_all_roots_in_json: include the class name as the root element for all contained objects (implies include_root_in_json), default false

include_root_in_json only puts the root element in at the top level, not on contained objects. That makes it compatible with the JSON expected when the matching flag is set in the from_json call.

PC vs. post-PC

To me the whole argument doesn't make sense. The world simply isn't going to dump what we consider the PC, it's just too useful. To me the argument boils down to one of form factor vs. usage:

Desktop. A standard large-case PC with a separate keyboard, mouse and monitor. It's advantage is, to quote Tim Allen, "MORE POWER!". It can run pretty much the most powerful CPUs around full-bore constantly without overheating. It doesn't worry about draining a battery. It can have a lot of screen real-estate with multiple large monitors. It can have a full-sized keyboard and mouse. It's got lots of relatively fast local storage, and continuous network access. It's nailed down to a single location and tethered to the network via a wire, which makes securing it easy. And while that's also it's biggest downside, the spot it's at is also where most of an office worker's work is anyway. It's most useful when you've got a lot of text to type, data to enter, output to view. It's, quite bluntly, the full-sized pickup truck or van of the computer world.

Notebook. Not the ultra-light, the totable one you carry around in a bag. You plop it down on a table, plug it in to power and maybe a network jack, and you're good to go. It's got a smaller keyboard, mouse and display than a desktop, and not as much horsepower under the hood, but it's ideal when you need something you can plop down anywhere but not something you have to constantly carry around. Think a businessman at a hotel, or someone out at a customer site for the afternoon. It's literally a portable substitute for the desktop, not a mobile device.

Netbook. This is the ultra-light. It's small, low-powered and easy to carry. Ideal for when you need to do some stuff that involves a keyboard but need something you can carry around all the time. It's what you might take to the coffee shop when you expected to be doing some IMing or light typing, but didn't expect to be typing page after page. Typically has a wireless network interface, and connectivity is assumed to be intermittent.

Tablet. Something from about the size of a paperback book up to a clipboard. No keyboard, no mouse, it's based around a touchscreen interface. You aren't going to be doing a lot of typing of text on this, the interface just isn't suited to it. But you can browse the Web easily, and deal handily with applications that involve selecting things from what's displayed on the screen. It's network interface is wireless, and it can't assume a constant network connection. Great for a lot of floor work like taking inventory, and times when you need to view the Web but not enter a lot of data.

Smartphone. Sizes up to about the size of a paperback book. It's mostly a scaled-down version of a tablet, the much smaller screen size being the big distinguishing feature. Where a tablet might be able to get away with the same sort of presentation you'd see on a notebook or desktop system, there's no way that works here. The other big difference is that, since it's expected to be connected to a cel phone network, it assumes a more constant network connection. It may take advantage of a wireless network if one's available, but if no it's still got a data connection through the cel network. Where a tablet's used to a greater degree for processing data and doing things, the smartphone form factor's used more for communication. Voice, video, IM, it's a pocket-sized device for talking to other people that oh, by the way, can also display data. A tablet, OTOH, is a device that displays and lets you manipulate data and oh, by the way, can also probably be used as a phone.

You can see how different the various categories are in terms of what people do with them. I think people don't want a replacement for the standard desktop PC, unless it's to replace it with a laptop that can accept attaching a larger monitor and a better keyboard and mouse for desktop use. And even then a lot of people would wonder why bother, since they don't want to take their desktop home with them. They either want something else instead of the desktop, because they don't do the things a desktop is good at and spend all their time doing things the netbook, tablet or smartphone form factors are good at, or they want those other form factors as an adjunct to the desktop with data shared between the various units. In large part what they want is the way Google services work: I can tie my Google address book and calendar into my e-mail program and when I update a contact there or through the Web interface the changes automatically show up on my phone, and vice-versa. If I change a document on the tablet, I want to see the changed version on my desktop system. I may want to make up my grocery list using a tablet as I check the cupboards and freezer for what we're out of, and then be able to call up that shopping list on the pocket-sized smartphone while I'm shopping and all I'm doing is scrolling through the list and marking things off as I get them (I could carry the tablet along, but it's a little big to fit in a pocket conveniently).

But in the end, I don't expect people will give up their big desktop systems for things like gaming or typing long documents or working with large spreadsheets or all the other stuff you do at the office. And I sure don't see PC gamers giving up their systems, there's just too much stuff you can do with a full keyboard and mouse that console games don't have the input devices to match. We'll be in a post-PC world in the same way IT is in a post-mainframe or post-COBOL world, or the way computers have led us into a post-paper world, or the way nuclear reactors and solar power have led us into a post-steam-boiler world. Oh, that's right, nuclear reactors and most large-scale solar farms just... boil water using a new heat source. Mainframes and COBOL are still firmly entrenched. And paper, well, "It's the 21st century. Where's my paperless office?". I don't expect "post-PC" to be any different. We'll add new devices and new capabilities, and the PC will adapt to work with them without ever disappearing (or even becoming less of a central player).

Microsoft dumping .Net for HTML5/Javascript

Mike James wrote a blog entry about this. My immediate thought: what did you expect? Microsoft sets their roadmap based on their business needs and what'll benefit them, not what'll benefit their developers. And Microsoft is focused on one thing: insuring that Microsoft controls everything. When they think "cross-platform", they mean "across all Microsoft platforms". They want it to be easy to have apps run on all their platforms, from desktops to phones, and to be as hard as possible to make those things run on any other platform. If things do run on other platforms, they want to insure that Microsoft is a required part of the infrastructure anyway.

C# and .Net and Silverlight are problematic for Microsoft because they can run on other platforms, and when they do other Microsoft products and platforms aren't required. It's not easy but the Mono project has demonstrated that you can create a C# compiler and .Net runtime for Linux, and once it exists for Linux it's a lot easier to port it to any other Unix variant. Silverlight will likely fall into the same category. But HTML5/JS has one big advantage: it's limited enough that you can't create an entire complex application in it. You can do the UI part, but you're going to need a back-end to handle things that just can't be done in Javascript. Microsoft's hope is to use the standardized nature of HTML5/JS to sell management by making sure the "standards-compliant" checkbox is checked, while doing some things under the hood and in the client/server communication with the back-end to insure that you'll need a Windows system in the mix to successfully run one of their new apps. Think back to ASP and how it got tied into IE-specific bits of Javascript and HTML to the point where it was hard to make ASP webapps work right anywhere but in IE. How successful Microsoft will be with this is up in the air, but I think their goal is what it's always been: to make sure it's Microsoft and Windows everywhere, for everything, to the exclusion of everything and everyone else.

As a developer, make no mistake: Microsoft does not care about you. You are important only insofar as you buying into their roadmap is needed for it to succeed. Once that's no longer needed you're at best unnecessary and at worst competition to be eliminated (every sale of software you write is the loss of a sale of something Microsoft wrote or could write, and you selling your software at the expense of Microsoft doesn't benefit Microsoft).

PS3 root signing key revealed

Apparently GeoHot has found and published the root key used by the Playstation 3 to sign and verify games. Not just the public key used for verification, mind you, which is the easy part. They've published the private key used to sign the game executables. With the private key, you can sign your own executables and they'll be accepted by the PS3 without needing any hacks, kludges or bug exploits. This is a big deal because the key's pretty firmly embedded in the hardware itself. A simple new firmware update can't change it. And if new hardware doesn't accept the old key, then all existing games simply won't play on the new hardware.

Frankly I don't see why the console makers are so bent on keeping anything but their approved software from running on the consoles. It makes no sense. Someone who wants to run, say, Linux on the PS3 still has to buy the full-blown PS3 console, there's no way around paying Sony for that. They may not buy any games if all they're interested in is running Linux, but then if they couldn't run Linux they wouldn't be buying those games either, nor would they be buying the PS3. They might use this ability to cheat at single-player games, but what's that hurt? They're still buying the PS3 and still buying the games, and there's nobody else to be affected by their cheating. Multi-player games... hacked games might be an issue, except that those games already have lots of ways of detecting cheating on the server side where it's safe from user intervention. For instance, to prevent target hacks simply don't send the client information about objects it can't see. Not even the best hack can target what doesn't exist.

In re Bilski

Monday is the last day for Supreme Court rulings to issue for this term. So far, no opinion in In re Bilski, the major patent case this term, has come down. Some people are thinking that it'll have to come down Monday, because the Court won't want it to hold over into the next term. PatentlyO makes that argument. They also make the argument that it'd be better for the appellant here to drop the case before the ruling issues, and that the only reason for the appellant to pursue the case is that they want business-method patents to suffer a setback. I think Crouch is wrong, Bilski is appealing only because it's the only way to overcome the setbacks they've suffered thus far (see the documents on the case at Groklaw).

Crouch does make an interesting point, though, and one that gives hope that the Court will uphold the denial of the Bilski patent and, by extension, support the Patent Office's new position that purely abstract things like business methods aren't patentable. That's that Monday is Justice Stevens' last day on the Court. He's also the only Justice who's short on delivered opinions, if he's writing the Bilski opinion it'd bring him right into line with the other Justices. If that's so, Stevens also has a track record in opposition to things like patents on abstract ideas and non-physical things. If he's writing the opinion, it's likely because the opinion was in line with his track record and not favorable to Bilski. This'd be good news for software developers. These days one major problem in software development are patents that are over-broad and vague, with their holders trying to apply them to everything in sight. Or patents on blatantly obvious or long-existing things like a shopping cart (but in a Web browser!). Between Bilski and KSR v. Teleflex, the courts and the USPTO have given opponents of over-broad patentability a lot of ammo. That's also another point in favor of the Court upholding the appeals court in Bilski, that'd be in line with it's thinking in KSR.

The alternative, of course, is that the Court decided to give Stevens a light load because he's retiring and the Bilski opinion will be held over for next term. But we can hope that's not the case.

Names

Yes, names. And the computer systems that handle them. If you write computer programs that handle people's names, read this blog post. Then read this article. Then go back and check your programs for how many of the assumptions in the article they make. Yes, all of those assumptions are invalid. Yes, you will have someone breaking them. Many someones. You'll have more people than you expect using your system. Think about this: right now if something occurs for one person in a million, you can expect more than 300 of them in the United States alone (307 as of July 2009).

And yes, someone out there undoubtedly has in fact legally changed their name to "Robert'); DROP TABLE users" just to be a prat. Your systems should be able to handle him in a suitably boring manner automatically, without needing special coding for SQL injection.

Signs you're abusing SOAP

Big red flag that you either don't know how to use SOAP or that you shouldn't be using SOAP for this job:

The only argument your call takes is a string, which contains XML which holds the structured data the server needs.

You should be either creating a SOAP object tree corresponding to the XML structure and passing your data as SOAP structured data, or you should be removing SOAP entirely and using simple HTTP POST.

System capacity

Immediate thought: if your system gets overloaded handling 20 requests a minute, something's really hosed. I'm used to thinking in terms of rates 10-100x that, and counting them per second not per minute. The problem of course is that the offending software is a commercial package, so we can't dig into it to find out what the problem is and fix it.

Verizon using mail submission port 587

Verizon is moving to using port 587 for mail submission, requiring encryption and authentication to send mail. That alone won't stop the spam originating from their networks, but it's a start. My thought is that there should be 3 ports for 3 different purposes:

• Port 25, no encryption or authentication required, is for server-to-server mail transfer. Relaying shouldn't be allowed, all e-mail arriving should be addressed to an in-network domain. Anything else should be rejected. This means no relaying. Messages should not be modified except for adding an appropriate Received header.
  
• Port 587, encryption and authentication required, is for end-user mail submission only. Mail submitted to it should have the Sender header stripped and replaced with one based on the authenticated username.
• Port 465, encryption required and authentication allowed, is a hybrid. If the session isn't authenticated, it should act per the rules for port 25. Authenticated sessions should be allowed to relay. If relaying, authentication information should be added to the Received header and if no Sender header is present one should be added based on the authentication information. Messages should not be otherwise altered.

One thing many ISPs ignore (often, I suspect, willfully) is customers who do not use their ISP as their mail provider. I'm an example. I get my Internet connection from Cox, but XMission in Utah host my domain and handle my e-mail.

Code comments

Motivated by a Daily WTF entry on code comments. I know exactly what creates this: Comp Sci instructors. You know them, the ones who insist that every line of code be commented, no matter how trivial. After a couple of years of this, students get in the habit of including comments just to have a comment for that line of code. Of course the easy way to do this is to just restate what the line of code does.

Now, as a professional programmer doing maintenance on code I don't need to know what the code does. I can read that line of code and see exactly what it does. I need something a bit higher-level. I need to know what the code's intended to do, and I need to know why it's doing it and why that way of doing it was selected. I know it's interating down a list looking for an entry, I need to know what that list is for and why the code's looking for that particular entry. Instead of comments describing how to iterate through a list, I need a block comment saying something like "We've got our list of orders entered this week, we know they're ordered by vendor, and we're trying to find the first order for a particular vendor so we can extract all his orders quickly.". Probably followed by something like "Now that we have this vendor's first order, we'll grab everything until we see the vendor number change. When we see that, we've got all his orders.". Much shorter, wouldn't satisfy that instructor at all since most of the code won't be commented, but much much more useful when I have to change things. It tells me what the code's trying to do, and what assumptions it's making that I have to avoid invalidating.

Too many Comp Sci teachers need some real-life experience maintaining the kind of code they exhort their students to produce.

Powerful and easy to use

"You'll find this system to be incredibly flexible and powerful. You'll also find it to be incredibly simple and easy to use with minimal training. However, don't ask it to be both at the same time."

Programmers and undefined behavior

ISAGN for a sadistic C/C++ compiler to school the current generation of programmers in the dangers of relying on undefined behavior. Too many of them do things like assume that dereferencing a null pointer should cause a crash and core dump. The problem is that nothing says that. The C++ standard leaves the behavior in that case undefined, which means the code and possibly the compiler is free to do anything it wants to at that point. It doesn't even have to consistently do the same thing every time.

So, a sadistic compiler. At run time it'd check for various sorts of undefined behavior. When it detected them, eg. an attempt to use the result of dereferencing a null pointer (as by calling a method through a null object pointer), it'd branch to a routine that'd randomly select from a list of dangerous things to do, such as spewing the contents of /dev/random onto the console, kill -9ing a random process on the system, or zeroing all blocks on a random attached storage device. In addition, the compiler would check for any undefined behavior it could feasibly check for, and take similar actions when asked to compiler such code. Thus, trying to compile "x = a++ - a++;" might result in your system disk being wiped.

The goal: to impress upon programmers that you cannot rely on undefined behavior. At all. Not on what it does, not even that it does the same thing all the time. The only guarantee you have is that you won't like what'll happen. So avoid it like the plague, or pay the price.

Zune lock-up bug

Owners of Microsoft's Zune MP3 player saw their devices lock up hard at the end of 2008. It turns out there's a leap-year bug in Microsoft's code. The clock in the Zune records time as the number of days and seconds since 1/1/1980. To convert that into a normal calendar time, the Zune starts with this code to convert the number of days to years and days:

year = ORIGINYEAR;
while (days > 365)
{
  if (IsLeapYear(year))
  {
    if (days > 366)
    {
      days -= 366;
      year += 1;
    }
  }
  else
  {
    days -= 365;
    year += 1;
  }
}

It's basically looping through incrementing the year and decrementing days by the number of days in that year until it's got less than a full year's worth of days left. The problem comes on the last day of a leap year. In that case, days will be 366 and isLeapYear() will return true. The loop won't terminate because days is still greater than 365. But the leap-year path inside the loop won't decrement the days because days isn't greater than 366. End result: infinite loop on 12/31 of any leap year. This bug should've been caught during standard testing. Leap years are a well-known edge case when dealing with dates, likewise the first and last days of the year and the transition from one year to the next are standard problem points where any errors tend to show up.

Microsoft's proposed solution: wait until sufficiently far into 1/1 of the next year, then force a hard reset of your Zune. Yeah, that'll get your Zune working but it doesn't fix the bug. Bets that Microsoft's fix for this code causes a different kind of failure on 1/1/2013?

JournalSpace mistakes mirroring for backups, dies

JournalSpace is dead.

Short form: JournalSpace depended on drive mirroring for back-ups. Something proceeded to overwrite the drives with zeros, and the mirroring politely overwrote the mirrors with zeros too. Their entire database is gone, all blogs, everything.

Repeat after me: mirroring is not a back-up. RAID and drive mirroring are for reliability and fault-tolerance. They'll protect you against hardware failure. They won't protect you against software doing something stupid or malicious. If the software says "Write this data to this location on the disk.", mirroring software and RAID drivers won't, I repeat will not, not write the data. If you're depending on your mirrors to contain something other than exactly what the main drive contains, well, you'll end up where JournalSpace is. You need point-in-time backups to external media, something that won't duplicate what's on the main drive unless and until you do the duplication yourself. That's the only way to insure that, if your software writes corrupted data to your main disks, the backups don't have the same corruption written to them as long as you catch the problem before you overwrite the backups with new ones. This is also, BTW, why you have more than one set of backup media: so if you do run your backups before catching a problem, you've got older backups to fall back on.

This should also be a cautionary tale for anybody wanting to host their data, application or whatever "in the cloud". If you do, and it's at all important, make sure you a) can and do make your own local backups of everything and b) have a fall-back plan in the event "the cloud" suddenly becomes unavailable. Unless, of course, you want to end up like the people who had their journals on JournalSpace: everything gone, no way to recover, because of somebody else's screw-up.

BodyParts webapp

I finished the first pass at the BodyParts webapp that lets me view and maintain the Legends&Lore body parts inventory for my guild in EQ2. Security framework enabled, anonymous viewing and a login for editing, Tomcat set up properly, Apache configured to front for it. Now I can update the inventory from the in-game browser, no more writing things down on paper and windowing out to update text files.

Next things on the list:

• Add add/subtract fields and buttons to the editing screen so that instead of having to do the math in my head I can just punch in the number of parts and hit Add or Subtract depending on whether I'm depositing or withdrawing parts.
• Move the inventory from an XML data file into a real database table. I'll want to move the user information into a database table in the process and change authentication to match.
• Reconfigure things to have all webapps in their own sub-tree so they don't sit directly at the root of the content tree. I'll have to see whether I want Apache to re-write URLs (so the user sees /webapp/BodyParts/ while Tomcat sees just /BodyParts/) or whether I want to make the change within Tomcat.
• Change things in Eclipse to not depend on MyEclipse for the Spring framework and supporting libraries. I'm finding that as I get more familiar with things it's easier to edit the XML directly than to depend on MyEclipse's tools.
  

It's been a useful excercise so far.

The Srizbi botnet

After being taken down when McColo was shut down, then resurrecting itself, the Srizbi botnet has been taken down once again. It won't last, though, the botnet will start searching domain names trying to reestablish contact with it's command-and-control network, and it's operators will probably succeed We need new tactics:

1. Instead of trying to block the botnet when it searches for a new C&C network, intercept it. Put something up at the next few domains it'll try that will respond correctly and take control of the botnet. Once you've got control, use that control to have the botnet download a new module that'll remove the botnet software from the computer, then shut down the computer until the user reinstalls Windows.
2. Start sanctioning users who allow their computers to become infected. Right now there's no significant penalty assessed against the people who continue to allow their computers to be used by the botnet month after month after month. Start penalizing them. If they get infected, their Internet access gets suspended until they can provide evidence to their ISP that they've cleaned their machine up. Second offense, they get a 3-month suspension. Third offense, they get permanently disconnected. It's the Information Superhighway, and just like the real road system if you accumulate too many points we take away your driver's license.

Keeping a computer secure takes some effort and thought, especially if you're running Windows which was designed to be vulnerable. If there aren't noticeable penalties for being insecure, users just won't put forth that effort.

BodyParts webapp

Slowly but surely I'm getting my head around writing a web app using the Spring framework. It's requiring a fair amount of work, but it's much easier to understand when I'm actually writing code and seeing it work (or, more often, fail miserably). I need to get error handling working, then adding a new species, and then adding the security bits to allow some users to edit and others to only view the data. Once I'm done I'll not only have a handle on Spring, I'll have a way to edit my guild's body-parts database while I'm in-game.

Projects I need to do

• Clean up LJBackup. It's been sitting on the back burner for way too long.
• Take the EQ2 body-part inventory system I've got, that's currently based on a hand-maintained CSV file, and convert it into a Spring MVC web app. For low-volume stuff I can run it under Tomcat on an odd port on my gateway machine. That'll let me update the inventory through the in-game browser while I'm actually moving body parts in and out. It'll also be a useful show-off project if I want to move into Java and/or web-app work.
• Create a utility to back up Blogger blogs, both entries and comments, locally. This should be easier than LJBackup.
• Multiple new computers. I've got the guts for a new gateway box (obsolete AMD dual-core Socket 939), just need a case for it. I can turn the existing machines into two boxes sufficient for the kids. That just leaves main/gaming systems.
• I need to scheme some way of getting a hosted server and the time to configure and maintain it correctly. I want to handle e-mail, Web server, DNS and such myself and drop the XMission hosting.
  

Computer models and the credit crisis

This is more political than normal, but it touches on tech so I'm putting it here. Basically, in the Congressional hearings, Alan Greenspan blamed computer models in large part for the mortgage and credit crisis. I'm sorry, but that's not so. The fault isn't even with the data fed to the models. The fault lies with the people in the banking industry who looked at the models and said "What do we need to feed these models to get the results we want?". They didn't just feed the models wrong data, they outright manipulated the data they were feeding the models to insure the models gave them a specific answer. That's always a recipe for disaster. They knew the models were telling them those sub-prime loans were bad bets, so they created "stated income" rules so they could feed higher incomes to the models and make the models say the loans weren't as risky. They created credit-default swap instruments that they could factor into the models to make the loans appear less risky, and then decided not to factor in the risk of those credit-default swaps also failing.

The banking industry decided what answers they wanted, then hunted around until they got models and inputs that produced the desired result. You can't do that with any sort of model. And if you do, don't blame the models and the techs for your failure to listen to what the models were telling you when that didn't agree with what you wanted to hear.

But then, we see that in IT from management all the time. The techs say "If we do X, we're going to see these problems.". Management berates the techs for being obstructive and standing in the way of doing X when the rest of the company's agreed on it. Then all the predicted problems occur, and management berates the techs again for allowing those problems to happen. PHBs in action.

Browser process models

Everyone's praising IE8 for it's new one-process-per-tab model. It's got many advantages over the threaded model used by most browsers, including the fact that a crash of one tab can't take other tabs or the browser down. What most people don't seem to get is that the multiple-process model isn't new with IE8, and in fact the threaded model was one adopted only over the strenuous objections of everybody else. You see, threads exist for one reason and one reason only: on VMS and Windows (which was designed in part by the guy responsible for designing VMS), creating a process is expensive. You need threads in those OSes because you can't create a lot of processes quickly. But Unix had cheap process creation from day one. You needed another thread in Unix, you forked off another process. Threads weren't needed. But everybody in the Windows community kept braying about needing threads and why didn't Unix have them, oblivious to the fact that they already had what threads did in the fork() call. So Unix finally, reluctantly, adopted threads with all their pitfalls. And programmers used them heavily when processes would've been more appropriate. Until the pitfalls finally became too much to live with, when they went from just driving programmers nuts to causing problems for the very users who demanded them in the first place. So now we're back to where Unix was 25 years ago, forking a new process when you need a new thread of execution.