This is a dumb one that took far longer than I would have liked.

SBS 2008. OWA was spitting out HTTP Error 500.19 – Internal Server Error

The requested page cannot be accessed because the related configuration data for the page is invalid.

It then listed information, owaauth.dll was mentioned somewhere. Should start taking screen shots of these… It also referenced a line of web.config from program files\windows small business server\bin\webapp\sbs web applications, a line containing only . The event log showed an error relating to the DLL as well:

The Module DLL C:\Program Files\Microsoft\Exchange Server\ClientAccess\Owa\auth\owaauth.dll failed to load. The data is the error.

And this one:

Could not load all ISAPI filters for site ‘SBS WEB APPLICATIONS’. Therefore site startup aborted.

The error on the website complained of something being locked.

Searches all talked about 32bit mode vs 64bit, unlocking strings, resetting IWAM and IUSR accounts (though they don’t seem to apply to SBS 2008?), recreating your OWA folders… I did all that for hours with no luck. In the end, it was permissions: owaauth.dll had Authenticated Users granted Read and not Read & Execute. I discovered this by comparing the file to another server. The whole OWA folder was jacked. IISRESET and we were back up.

The moral here is to always remember to check your basics first. Two of my co-workers looked at this for well over an hour before it was passed to me and I won’t say how long I spent working on it before I found this. If the error says that a file can’t be loaded, maybe it actually means that the file can’t be loaded.

It is midnight and I promised myself I would go to sleep an hour ago. That promise seems to be the one I am the worst at keeping, right up there with flossing, exercising every day, regular oil changes, and basically all of those things that we’re told responsible adults do when in reality they exist just to make us feel like losers. Or something like that. Regardless, it’s midnight and though I want to sleep, I also want to get this out while it’s still somewhat fresh in my mind.

I have a love-hate relationship with Symantec. Funny, I feel like I made that statement before… Oh, that’s right, I did back in February when BESR was BSOD-ing my servers. Well it’s still true and no matter how hard it hits me, I keep coming back with hope that next time, this very next time, things will be better. Of course, they never are; those sneaking suspicions you have in the beginning of a relationship, those red flags telling you to get out because it’s just not going to work — they’re always right.

I have been wearing rose-colored glasses when looking at Symantec Backup Exec for years. When it works, it is great. Lots of features, an interface that does make sense once you spend a few minutes with it, support for granular restore of all sorts of things, works well with all kinds of different media, not really prone to crashing… As long as you aren’t trying to backup something that makes use of its Advanced Open File Agent. Oh, and Exchange might give you weird errors. And Hyper-V? If it doesn’t give you VSS errors, you’re still limited to full backups of your VHDs only because it doesn’t operate at a block-level.

The problems with Backup Exec are staggering. We have installed it on dozens upon dozens of servers and the amount of hand-holding required to make it do its job consistently just blows my mind. To Symantec’s credit, their tech support staff is usually very knowledgeable and will try their best to resolve and educate, but nothing changes the fact that it is the rule, not the exception, for VSS errors to just appear out of nowhere, require a reboot, and then everything will just work. The knowledgebase articles linked from the event log are often completely unrelated to your issue, resolutions to simple issues are elusive… What the hell is going on? While our most simple servers, usually stand-alone DC/File/Print servers without Exchange, Hyper-V, or SQL seem to work consistently, get a multi-server network, add in a few roles, maybe a USB drive, and you are fucked.

So with the taste of Backup Exec 11 through 2010 R2 (I haven’t played with R3 yet) still reminding me of something between orange juice + toothpaste and cat urine, I was delighted to start using Backup Exec System Recovery 2010 and the new Symantec System Recovery 2011. As I described in my post back in February, this application, which is Norton Ghost with a different name and some extra features intended for enterprise users, simply works. Always. I guess because it works at a block-level, there is less that can go wrong? Whatever it is, install is a breeze, setup is a breeze, backup and restore are fast, granular restore works well, technical issues (aside from that BSOD thing…) have not been bad, the P2V/V2P options are great, and the price is fantastic.

The pricing for the virtual edition of Symantec System Recovery 2011 is very attractive. For something like $2000, you can install on a physical host and unlimited Hyper-V VMs. This appealed to me and a client whose Backup Exec-protected Hyper-V cluster was a shitstorm of negativity. While we provide help desk support and try to stay attentive, we are not there full-time and cannot afford a backup software that requires constant hand-holding so it does its goddamn job. I downloaded and installed SSR 2011’s trial, a 350 MB-ish application. Install was easy, configured a backup, configured offsite backup, created my first image — awesome. And then I decided to install the management “solution.”

And that’s when things got bad.

The Management “Solution” is something like 1.8GB in a single ISO. I downloaded it simultaneously to my client’s machine, a Server 2008 R2 Enterprise Edition box and my company’s server, same version of the OS. I made sure to download what was indicated as the 64-bit edition. A short time later, I extracted the ZIP, ran the Setup executable on my client’s machine and… got an error that it wasn’t a valid 32-bit application. What the fuck? I tried the browser application in the same directory, same error. I tried it on my server — different but similar error. Immediately. We were not off to a good start. I dug through the folders a bit and buried somewhere in there I found the installer for the 64-bit version of the program, which I discovered was the Symantec Installer or something along those lines. I installed the installer, fired it up, and about 90 minutes later had installed the prerequisites for the management utility which included SQL Express and the IIS role, then the various components that made up the software: Altiris Management Console 7.1, the installer for SSR 2011, the SSR 2011 remote agent, and the SSR 2011 Management Console.

I’m going to first whine about this installation process. The fact that I had to jump through hoops is one thing, the fact that it took about 90 minutes to install this software is another; the fact that I am installing a very sophisticated management application to remote control NORTON fucking GHOST: SERVER EDITION is too absurd for words. The fragmentation of Symantec’s product lines is unreal. Symantec Endpoint Protection has its own console that’s pretty cool, Backup Exec has its own management utility, Brightmail Security looks exactly like SEP (and correct me if I’m wrong but I think the two integrate?), and then I am installing the largest, most cumbersome, complex utility to control what is their most straightforward, functional, “set-it-and-forget-it” business backup app? Are you kidding me? We’re going to come back to this.

I’m going to skip the rest of the story and cut right to the facts.

After the installation was finished, I was unable to login to the SSR 2011 console because the shortcut the application created was formatted incorrectly. Yes, immediately after the installation, I couldn’t get into the application, the huge application that took forever to install, that made me hunt for its executable. The reason was idiotic: you manage this software through a web browser. There is an option to manage the application through SSL and I set the port to the default, 443. After troubleshooting for far too long, I discovered that the URL looked like this:

http://localhost:443/Altiris/blahblahblah

But… 443 isn’t HTTP, it’s HTTPS. I modified the shortcut to be https://localhost/altiris/blahblahblah and it loaded right up.

I spent some time looking at the documentation. The interface is clunky, figuring out exactly how to find a machine and deploy the agent and then the software took far too long and didn’t seem to want to discover anything other than the local host. It didn’t want to deploy the SSR 2011 Remote Agent and it was telling me that the application was not installed, something I attributed to the installation of the software first and then the management console. If there is a next time, which I hope there isn’t, I will do the management console first. After some fighting, I got the remote agent installed locally.

The SSR 2011 application would not accept my license key. I mean, it would kind of accept it after a 2 minute+ hang, it would say that it was OK, but then I’d check Help -> About it and it would say trial. The management console said the same thing.

Changes to the backup policies were complicated to apply. Changes to just about everything were hard to apply.

With all this complaining, I want to make it clear that figuring stuff out is my goddamn job. I get paid pretty well for it, I consider myself pretty good at it, I really like doing it. I am confident that if I had sat there long enough, fighting with this software, I would have made more sense of it. I am sure that it is powerful and has all kinds of great features. That’s nice and all but the fact is, a simple, straightforward application like Norton Ghost should not have a complex management utility that plain doesn’t work. We’re going to come back to this when I’m done describing how shitty the software is.

I uninstalled SSR 2011 manually so I could push it down from the console, thinking that might correct my license issue and other quirks. Well, the software wouldn’t install from the remote agent. The Windows Event Log showed an IIS error relating to ASP.NET permissions, something I was able to find someone else describing in a thread on Symantec’s forum when they were beta testing Altiris 7.1. Again, what the fuck? That’s it? I messed around with it a bit, tired, hungry, and frustrated. It got worse, I uninstalled and reinstalled, it failed. I threw my keyboard, sent a defeated email to my co-workers who were telling me that the app would suck from the get-go, and went home. As soon as I have a moment, I will call the client, tell him that the software was junk, recommend AppAssure (Kaseya integration, competitive price, fantastic features, NOT MADE BY SYMANTEC), and manually rip out the pieces of these products that did not want to come out on their own.

That is my SSR Management Solution Experience but that is not the whole of my rant.

Growing up with Windows, I think that many of us learned to expect that things wouldn’t work. There are these paradigms that we except as laws, things like “new software will have bugs,” “sometimes you just need to reboot to fix a weird error,” “powerful software is inherently complicated.” It’s kind of like living in a third-world country that doesn’t have clean running water without ever knowing how we live here in America. You just don’t know so you think it’s normal.

Linux changed that for me. I remember how shocked I was when I started getting into basic web app development and configured my server. I needed to install some prerequisites and it told me what they were and asked if I wanted to download and install them. I hit Y and Enter, it installed them, and that was that. My server can stay up for weeks, months, without ever needing a reboot. Things don’t crash, they don’t freeze. My backups just work. Apple has proven this to us as well with their desktop OS and iOS. So why is it still acceptable for Windows applications made by large corporations and intended to provide crucial protection for businesses so inconsistent and unreliable? Is it Windows? I don’t know. Windows 7 works pretty well and so does Server 2008 R2, though we still have to reboot after making most system configuration changes

At the risk of talking about things that I don’t understand, I feel as though big companies feel as though they have the IT industry by the balls when it comes to certain applications. They know that because the expectations are so low, they only have to be just a little bit better than the other guys to be ahead of the curve. Backup Exec performs so inconsistently that when a new version is released and it STILL has the same stupid VSS problems as the last three versions, we will be a little frustrated but shrug it off — that’s just what VSS does. Except it’s not, as shown by so many other backup utilities. Symantec System Recovery and its management “solution” are new, so we can expect a few bugs right away. I mean, that’s why we don’t install Windows Updates or Service Packs until everyone else has tested them first, right? I call bullshit.

I find that there is a double standard when it comes to treatment of end users. Consumers are babied because there are more of them and they are seen as stupid because, well, most people really are stupid. Because of that, Symantec made sure that Norton Ghost was simple, efficient, and easy. When they ported it to servers, it was already all cleaned up. The same goes for the client-side version of SEP: clean, easy, unobtrusive. But IT professionals? They know that we are used to shit just not working. They know that we expect to have to struggle because we’re supposed to be cool and patient, we fix things and deal with problems for a living so we expect there to be rough edges… and I think it is absolute garbage.

An anecdote to illustrate:

Google. I like Google Apps. Features, price, presentation. A great alternative for small businesses to Exchange. We have been selling it so I signed us up to be resellers.

The Google Apps Reseller signup process works like this: you apply. You prove to them that you can sell the product by recruiting something like 20 users for them, basically as a test. When you’ve done that, you submit your information and you wait. And you wait. And you wait. There is no indicator of where your company is in the queue, no way of checking on your status, no automated, “Hey, we haven’t forgotten about you, we’re still processing applications and we appreciate your patience” emails. Their website has no contact information other than a reseller signup forum which, as you can guess, is filled with people asking “What is my status?” Everyone waits over a month, no exceptions. If you complain, they escalate you.

In my case, I complained publicly on their board that I not only had to wait a month with no response but their attitude was disrespectful and un-Google-like. For a company built on friendly, simple, functional technology to ask potential salespeople to sell their products on good faith and then accept a response of, “Good job! Now don’t call us, we’ll call you!” is insulting. I sold the product on good faith and I deserve for my time to be respected. I should not have to bitch publicly to find out that they were having trouble getting information about my company because of some typo or something, some intern should write a little interface that says my status. Quick and easy. And you know what? A Google spokesperson responded immediately, apologized, agreed, said that his hands were tied and they would love to provide more information but their development staff was focused on adding features for resellers. The signup process was low on the priorities list.

That sucked but it’s over; my point is that these double-standards show what these giant corporations think they can get away with. Symantec is far worse, with their fragmented, barely-functional, over-engineered, bloated applications.

I resent the idea that technology has to be complicated to be powerful. The problem I see is that so many companies have built industries around their software having kinship to archaic magickal arts, rituals known only to a select few who have devoted their lives to studying and practicing, that if they were to suddenly release a new version that was built for better usability, people would be out of jobs and pissed off. If Cisco’s web interfaces were suddenly more like SonicWall’s, they would lose all that money coming in from certifications and sooner or later, people would be out of jobs because there would be no need for them, the elevator and telegraph operators of the modern era. We tolerate cumbersome, clunky interfaces because that’s what we’ve always known and for some, buried within that is job security, it is profit… but it is also elitism, it also keeps powerful technology out of the hands of those who lack the funding to pay for experts with the knowhow to operate it. It ensures that people who have devoted their life to feeling superior for spending the time to learn this technology won’t have to learn new skills and stay competitive in the market.

And this takes us back to Symantec System Recovery 2011 and its Management “Solution.” At the end of the day, I was left with a management console that was the nuclear reactor to my backup software’s battery-powered flashlight. Why would software that screams for use by small businesses, priced cheaply enough to put it in the hands of just about anyone, have a management console with a confusing installation process, a ton of extra server roles and prerequisites required, a BROKEN shortcut immediately after installation, errors in the event viewer, and a clunky-as-all-hell user interface? If I can setup a Hyper-V cluster and write troubleshooting blog posts that generate a shit-ton of traffic to my site every single day, I can figure out the quirks of a management console and make it work… but why should I have to? Why am I expected to? Because we have an enterprise IT culture that tolerates shitty software, a corporation that buys smaller companies and develops parallel versions of redundant software, and a corporate software development culture that knows we have low standards so they have absolutely no reason to change.

When I bought a Royer R-121, I added to my arsenal what is widely regarded as the best microphone for electric guitar of all-time.

When I acquired my API preamps, I gained the coloration found on an endless number of classic recordings from the past fourty years.

When I assembled my new PC, an Intel i7-950 with 12GB of DDR3, I was no longer limited by my computer and the sky became the limit.

But one thing still nagged me. With all of my high-tech gear, I was missing one integral piece of a professional studio: tape. The harshness of digital recording is like a shotgun to my eardrum and no matter what I do, that cold shrillness of something sounding through my monitors the way it actually sounds when standing in front of it makes my blood boil.

So I fixed it.

I would like to introduce you to a piece of classic gear. An item that has been owned by countless classic producers, an item integral to the history of recording. Friends, I would like to introduce…

The Tascam Porta02. No balanced connections, no phantom power, no bullshit. Just right.

OK, so with that out of the way, I’d like to say how much the whole attitude of “tape, good; digital, baaaad” bugs me. I can’t begin to tell you how often I hear or read someone remark, “Oh yeah, we’re recording here. He uses all analog shit, man. It’s fuckin’ great,” or, “You record digitally? That’s cool but tape is just so much better.” Seriously, shut up. Just shut up.

The notion of tape being better than digital recording comes from a few places, as far as I’m concerned. Here they are.

One, home recording has fucked up recorded music. While it is a wonderful equalizer and allows young bands and individuals the ability to record and share their music, the accessibility of cheap gear and guys who setup their Digi 002 and suddenly have a studio causes a real problem for guys who invest thousands in their setup. Well, maybe not a “real” problem but it is an inconvenience and frustrating when good bands sell themselves short by going with cheap instead of good… but that’s another story. Anyway, the point is that because of the cheap AD converters you find floating around, my generation thinks that is what a digital recording sounds like, doesn’t really consider the whole AD/DA process, and don’t stop to think that just as there are different grades of guitars and drums and cars and phones and dogs and fucking people there must be different grades of analog-to-digital converters and tape machines. Those of us who come from the DIY scenes and grew up listening to noisy, shitty recordings discovered vinyl and realized that it made all that noise at the top and bottom feel much softer, so it’s easy to follow the logic that says that if vinyl is better than CD (which also pisses me off) than tape must be better than digital.

Second, tape has this coolness to it that it gets just by being old, slow, and cumbersome. It’s also expensive, which helps too. It’s also becoming more of a rarity. Bonus! We want what nobody else is doing.

Third, tape really CAN sound better. In some cases. Allow me to educate, studio-shopping friends.

First of all, in the right circumstances, tape is awesome. All tape colors sound in a unique way. It also does interesting things to dynamics. If you hit tape hard enough, you get natural compression that is incredible. You get warmth and punch, shrill highs are softened, lows fill out. There is a vibrancy and fullness that is instantly recognizable as coming from tape that is impossible to get in a PC unless you are using the UAD Studer plug-in (which is debatable) or the Nebula R2R library combined with the same developer’s TB+ and VTM-M2 which, by nearly all accounts, does a flawless job of making signals passed through it sound identical to what it would be if sent through the actual hardware. It is amazing. But that’s not my point here.

What is my point is that like anything else, there are different grades of tape and there are different grades of tape machines. What many musicians and uneducated studio engineers do not seem to realize is that a cheap tape machine, while having a sound that might be worth using in some cases, is going to sound cheap and shitty. If you don’t keep it maintained — if you don’t keep ANY tape machine maintained — it will sound worse and worse over time. Additionally, the actual tape you use is going to affect the end result. I’m no expert on tape machines or tape brands, but I can tell you that if you are looking for a place to record and you really want to use tape, you need to find out the model of the machine, what kind of maintenance is performed, and what kind of tape they’re using. Research the gear, get samples of their previous work, and listen carefully to it. After that, find out what they do after it’s on the tape and if they mix 100% in the analog world or if it at any time goes into a computer. Because if it goes into a computer…

…digital recording is not the same everywhere you go. The analog-to-digital process is not universal and your $500 converter is not going to sound the same as your Lynx Aurora. Same mics, same cables, same preamps, same room, same instruments, and your recordings will sound different. It might be subtle, the kind of thing you don’t notice when soloing tracks, but it’s usually easiest to pick out in the stereo image and extreme high and low frequencies and something that is noticeable when listening to a full band recorded using cheap AD compared to expensive AD. Cheap conversion is one of the hallmarks of a hastily-assembled studio. The guy with the killer mics into the OK preamps using shitty cables into low-end converters and then coming back through entry-level monitors: that’s someone who needs to read more. But he is everywhere.

So what I see happening way more often than I’d like is this: good band is attracted by a studio’s analog gear. They go and record there and the guy is using a tape machine but… it sounds kind of crappy, kind of… weird. But because the band has been telling themselves and been told that old = good and vintage = cool and lo-fi = awesome, they’re all afraid to say, “Hey, wait, my $2000 guitar played through my $2000 head into my $1200 cab doesn’t sound the way it does when you’re standing in front of it.” The Emperor’s New Clothes: audio engineering edition. Then, to make matters worse, the studio just uses the tape for that “vintage warmth” and then they dump it into their PC to edit it by sending it through their Digi 002. Hell, even if they use a killer Studer tape machine, as soon as they send it through those cheap preamps and that cheap converter, their quality immediately decreases. But if anyone notices, they don’t want to say anything.

Like tape machines, you can spent tens of thousands of dollars on AD conversion. The catchall “digital recording” is unfair to those who invest in Mytek gear, just as calling my Tascam Porta 02 “vintage tape” is an insult to those whose carefully maintained Otari machines are the pride of their studios. My point here: know what you’re looking for, know what you’re getting. The idea that digital recording is inherently cold and brittle is just as absurd as the idea that all tape is warm and punchy.

With Harold Camping and Friends’ PR stunt coming to a close in fewer than 24 hours, the project has generated the buzz we can only assume he hoped for. The internet is on fire with conversation, Facebook might as well be the Official Doomsday Discussion Forum, CNN has it on the front page, many of the top websites found by searching “May 21” are inaccessible as a result of increased traffic. I had dinner tonight with my parents — I haven’t seen them for months — and the absurdity and irresponsibility of those involved with this stunt was the big topic of conversation. We, the non-believers, are getting a good laugh, a well-deserved sense of superiority, mocking or maybe feeling some sort of pity for those who destroyed their lives because someone told them that nothing would matter past this Saturday. We are shocked, confused, maybe even a little nervous about the fact that these people went to such lengths to support something so completely ridiculous. Thankfully, they will soon go away and we can get back to life as normal. Right? No.

Harold Camping and his cult are fascinating but this level of blind faith can be found within millions of Americans. The only difference between the people who are excited for the Rapture tomorrow and those who know without a doubt that the world was created in seven 24-hour days is that Camping’s people have picked a specific date while everyone else is just kind of waiting around. When this doomsday cult fades away, the more sinister cult, the one with better funding, better organization, better PR media, and influence over school boards and every level of government will still be there, plotting and working towards their goals. These people have a freaking theme park, for crying out loud — a brainwashing machine aimed at children, slick and polished, all to reinforce the idea that the Bible is the word of God, evolution is wrong, the world is only a few thousand years old, and secular society is sick and evil. Given the opportunity, they would have us submit, despite the fact that irrefutable science is against them. I am not saying that everyone who interprets the Bible literally is a political activist bent on subverting our secular world; I am, however, saying that there is a huge movement of those who are.

I hope this media circus reminds everyone of just how crazy religion makes some people. As this fades away, try to remember that the same blind faith that fueled this doomsday cult fuels another doomsday cult, one that can’t predict the end so it’s content with simply taking over. Be aware, talk about what it means to you, get organized, and actively reject Christian extremism wherever it is found.

Immolith began recording their new full-length yesterday. They came by Friday to load in and do scratch guitars to a click so Saturday, we were able to get drum sounds and go.

I can’t begin to express how happy it makes me when bands record to a click. The whole process is faster, smoother, and the end result is tighter.

This is my first session with the new preamp. It’s cool but I ended up using sending snare top through the Pre 73 and that fourth API channel sort of ended up not mattering. That isn’t to say that the bottom snare (which went through the VP26) doesn’t matter but I don’t think the world would have ended if it had gone through the board. Oh well. An extra channel can’t hurt.

We’re finishing up guitars right now. I’m amazed at what the Carnhill transformer did to the Pre 73. I never quite understood how people used a single 57 for distorted guitar until now, though the R-121 still adds quite a bit of body to the sound and I wouldn’t give it up for the world.

Something exciting happened yesterday: I found a potential new apartment and studio space not far from where I live now. I don’t really want to say much about it beyond that but it is very promising!

That’s it for now. Back to unholy black metal with Immolith!