Art & Science

Fire & Ice... Day & Night...

This question, Art vs. Science, has come up a million times in software development circles. Reading Paul Johnson's (Paul's Pontifications) blog post, in conjunction with a discussion in the Tech Mill at Edgewater, (thanks, Jason!) I have come to see that art and science are not as opposite as I once viewed them to be.

What hit me was that Paul makes the statement that there's no process to implementing software. I still disagree. There are many processes.

The number of processes that an implementer can choose from to write his/her code is often vast, and depends on the problem set. A problem set includes many things, including requirements, tools, target platform, development platform, existing code, and even the implementer's mood and frame of mind. That is what makes implementing code, like painting, or creating a recipe, an art.

Within a common implementation problem set, there can be a large number of processes which can be applied to derive valid solutions. In fact, there are so many, that some distinct processes may actually render the very same code. So, to be more clear, it's not that there's no process... it's that there's no single valid process.

Knowing that there's no one single valid process doesn't mean that we can't pick a needle from the haystack... if the process produces a solution within the problem set, it's good.

Now consider what happens when you start to narrow a problem set. There's lots of things you can do. Frameworks, platforms, clear-specific requirements, best practices, coding standards, well structured architectures... these things are all factors that limit the problem set. By narrowing a problem set, you narrow the number of valid processes. By narrowing the number of valid processes that a developer can choose from, lots of interesting things start to happen. You achieve more predictable results, and are more likely to achieve repeatable schedules... and you reduce overall project risk.

This is what's so interesting about contemporary trends in software development, such as Ruby on Rails... use of these tools narrows problem sets that developers face. This means the implementer can spend less time figuring out where the blanks are, and more time filling them.

Now let's take this further. What happens when you reduce the problem set dramatically...? Take a single, relatively well known problem, on a very specific platform, using a very small set of unambiguous expressions. You get a very tightly defined process. By doing this, you wring the art out of creating something, to the point where it becomes machinable. The process becomes realized as a factory.

So to answer the question... Art or Science?

It's a trick question... art and science are not exclusive opposites. Art is about freedom to choose your creative process. Science is about knowing what processes are available, and the pros and cons of each. So programming, like all creative activities, is usually art (except in single-processed cases), and usually science (except in cases of serendipity and true miracles).

Paul's Pontifications: An Under-Appreciated Fact: We Don't Know How We Program

Multiprocessing: How 'bout that Free Lunch?

I remember reading an article, a few years back...

The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software

Its tagline: "The biggest sea change in software development since the OO revolution is knocking at the door, and its name is Concurrency."

Mr. Sutter's article suggests that because CPUs are now forced to improve performance through multi-core architectures, applications will need to typically employ multi-threading to gain performance improvements on newer hardware. He made a great argument. I remember getting excited enough to bring up the idea to my team at the time.

There are a number of reasons why the tag line and most of its supporting arguments appeared to fail, and in retrospect, could have been predicted.

So in today's age of multi-core processing, where application performance gains necessarily come from improved hardware throughput, why does it still feel like we're getting a free lunch?

To some extent, Herb was right. I mean, really, a lot of applications, by themselves, are not getting as much out of their host hardware as they could.

Before and since this article, I've written multi-threaded application code for several purposes. Each time, the threading was in UI code. The most common reason for it: to monitor extra-process activities without blocking the UI message pump. Yes, that's right... In my experience, the most common reason for multi-threading is, essentially, to allow the UI message pump to keep pumping while waiting for… something else.

But many applications really have experienced significant performance improvements in multi-processor / multi-core systems, and no additional application code was written, changed, or even re-compiled to make that happen.

How?

• Reduced inter-process contention for processor time
• Client-server architectures (even when co-hosted, due to the above)
• Multi-threaded software frameworks
• Improved supporting hardware frameworks

Today's computers are typically doing more, all the time. The OS itself has a lot of overhead, especially Windows-based systems. New Vista systems rely heavily on multi-processing to get performance for the glitzy new GUI features.

The key is multi-processing, though, rather than multi-threading. Given that CPU time is a resource that must be shared, having more CPUs means less scheduling collision, less single-CPU context switching.

Many architectures are already inherent multi-processors. A client-server or n-tier system is generally already running on a minimum of two separate processes. In a typical web architecture, with an enterprise-grade DBMS, not only do you have built-in “free” multi-processing, but you also have at least some built-in, “free” multi-threading.

Something else that developers don’t seem to have noticed much is that some frameworks are inherently multi-threaded. For example the Microsoft Windows Presentation Foundation, a general GUI framework, does a lot of its rendering on separate threads. By simply building a GUI in WPF, your client application can start to take advantage of the additional CPUs, and the program author might not even be aware of it. Learning a framework like WPF isn’t exactly free, but typically, you’re not using that framework for the multi-threading features. Multi-threading, in that case, is a nice “cheap” benefit.

When it comes down to it, though, the biggest bottlenecks in hardware are not the processor, anyway. The front-side bus is the front-line to the CPU, and it typically can’t keep a single CPU’s working set fresh. Give it a team of CPUs to feed, and things get pretty hopeless pretty quick. (HyperTransport and QuickPath will change this, but only to the extent of pushing the bottle necks a little further away from the processors.)

So to re-cap, to date, the reason we haven’t seen a sea change in application software development is because we’re already leveraging multiple processors in many ways other than multi-threading. Further, multi-threading options have been largely abstracted away from application developers via mechanisms like application hosting, database management, and frameworks.

With things like HyperTransport (AMD’s baby) and QuickPath (Intel’s), will application developers really have to start worrying about intra-process concurrency?

I throw this one back to the Great Commandment… risk management. The best way to manage the risk of intra-process concurrency (threading) is to simply avoid it as much as possible. Continuing to use the above mentioned techniques, we let the 800-lb gorillas do the heavy lifting. We avoid struggling with race conditions and deadlocks.

When concurrent processing must be done, interestingly, the best way to branch off a thread is to treat it as if it were a separate process. Even the .NET Framework 2.0 has some nice threading mechanisms that make this easy. If there are low communications needs, consider actually forking a new process, rather than multi-threading.

In conclusion, the lunch may not always be free, but a good engineer should look for it, anyway. Concurrency is, and will always be an issue, but multi-core processors were not the event that sparked that evolution.

Corporate Specialization

There's an old adage: "Anything you can buy, you can produce yourself for less."


In our business, we're well aware that there are a few fundamentally flawed assumptions with that sentiment. Despite the barriers to entry and many other recurring costs, somehow the idea seems pervasive in the business world.


I started my career in a consulting shop that insisted it was a product company. Then I moved, through market forces, into products based companies. I stayed on board with some of them long enough to help shut out the lights and unplug the servers when the sales didn’t hit the marks. The other big problem I’ve seen with product shops was that it’s engineering group typically went through its own “release cycles”… once the product was released, my team was cut to the bone.


I've never been in a classic IT shop, per se, but I’ve definitely been on tangent IT-support projects within “product” oriented companies. In IT groups, I’ve often thought that companies might see IT projects as frills and overhead. At some level, I think the pervasive IT-alignment is a counter measure to that idea. Still, it seems IT projects are typically viewed as liability, rather than asset. When it comes time for the company to refocus on its core competencies, (which is inevitable in the ups & downs of business) IT projects are prime targets for cutbacks.


Since the “.COM bust”, in these types of companies, an engineer on the staff for just three years is often seen as a “long-timer”… but no one feels bad about that, since a lot of these companies fold in that time frame, as well.


After experiencing the product based companys' ups & downs, and seeing many colleagues who had already been through the IT shops, I’m convinced… the outsourced project / consulting route is really the wave of the future, even more than it has been in the past. It’s only a matter of time before business admits this necessity as well.


It makes sense... I wouldn’t figure on every company to own & operate their own power plant.



Why should they typically want their own software engineering group if they don't have to?

[Edit: See also Part 2 Here]

The Wetware Bus

A colleague pointed me to this post about display size improving productivity.

It brings up a favorite topic of mine... With my current laptop (@1024x768), it's a bit of a pet-peeve.

It totally makes sense that tuning bandwidth to wetware improves productivity. Display real-estate is the front-side bus to wetware!


That's why I also like the Dvorak keyboard layout... Personally, I never achieved the level of proficiency with twenty +/- years of QWERTY (with formal training) that I have with the Dvorak layout over the past three years. Dvorak's a hard curve to get past, but I committed to it in the interest of tuning bandwidth (and maybe hoping to reduce RSI).

All Exceptions Are Handled

A recent colleague of mine was fond of pointing out that “all software is tested.” This truism is a simplification, of course, but the basic premise is this: the bugs will be exposed. The only question is “by whom?”

I submit for your consideration another truism, perhaps a corollary to the former: “all exceptions are handled.” Yes, ALL exceptions… and I’m not just talking about a top-level catch-all exception handler.

That requires further explanation. In order to fully back that up, let's talk about what a typical exception handling mechanism really is.

According to Bjarne Stroustrup in his classic (“The C++ Programming Language”), “the purpose of the exception-handling mechanism is to provide a means for one part of a program to inform another part of a program that an ‘exceptional circumstance’ has been detected.”

Unfortunately, Stroustrup’s book was published a decade ago, and the consensus on the subject among software engineers has hardly been less vague since.

Really, the typical exception-handling mechanism enables a delegator to effectively reconcile an expression of non-compliance in its delegated tasks.

So exception-handling mechanisms usually have several parts. To begin with we have a task or process. Then, there’s a task supervisor. The supervisor is the method that contains the typical try-block. This try-block generally wraps or makes calls to the task. There’s the exception object, which is really the information that describes an occurrence of non-compliance… the excuse, if you will. An instance of an exception object is not the actual exception itself, but merely the expression of the actual exception. Finally, there’s the handler, which is the reconciliation plan that is hosted by the supervisor.

In many languages, supervisor code can have zero or more reconciliation plans, each for a particular class of exception. A handler can also throw or re-throw, meaning a succession of supervisors could potentially be informed of an insurrection.

So it can be said that there are actually zero or more reconciliation plans or handlers… within the program. How do “zero handlers” square with the bit about all exceptions being handled?

We developers easily forget that software is always a sub-process of another task. Software doesn’t run for its own entertainment. Software is social. It collaborates, and blurs the boundaries between processes. Software processes that are not subordinate to other software processes are subordinate to… (I’ll borrow a term) wetware. When software experiences a circumstance that it cannot reconcile, the software will invariably express non-compliance in some fashion to a supervisor. In many cases, the supervisor is appropriately software. In many other cases, the supervisor is wetware.

Regardless… whether the program goes up in a fireball of messages, locks hard, blinks ineffectively, or simply disappears unexpectedly; these acts of insubordination are expressed to wetware. Wetware always has the final reconciliation opportunity. A user can take any and all information he or she has on the subject of the exception to handle that non-compliance.

The user could do a number of things: They could report it to their superior. They could ask someone else to investigate. They could try again. Even if they choose to do nothing, they are making that “informed” decision… and there. That’s what I mean when I say “all exceptions are handled.”

Now don’t worry. I’m certainly not about to advocate unprotected code.

Unfortunately, we developers aren’t trusting types. We go out of our way to protect against exceptions everywhere. We write code in every procedure that secretively skips over processing something if there’s a broken dependency. We write exception handlers that quietly log the details or completely swallow exceptions.

There are occasions when “do nothing” is a reasonable reconciliation. Much of the time, however, we engineers are engaging in illicit cover-up tactics. Handling decisions get co-opted from real process authorities and made at levels where the best interests of the stakeholders are not considered. Often cover-ups add redundant checks that make code harder to understand, maintain, and fix. It actually becomes complexity that exceeds that of proper exception handling.
Where does this get us? You gotta be fair to your stakeholders.

Keeping all stakeholders in mind is critical to building an exception throwing and handling strategy, so let’s dig in on that a bit. The first stakeholder most everyone thinks of in just about any project, is the project sponsor. The business requirements most often need to support the needs of the project sponsor. The end user comes next, and things like client usability come to mind. Sometimes end users fall into categories. If your program is middleware, you might have back-end and front-end systems to communicate exceptions with. In some cases, you not only have to consider the supervisory needs of the end user, but also, potentially, a set of discrete back-end system administrators, each with unique needs.

Remember, however, that during project development, you, the developer, are the first wetware supervisor and stakeholder of your program’s activities. You’re human; you’re going introduce errors in the code. How do you want to find out about them?

Here’s a tip: the further away from the original author that a bug is discovered, the more expensive it is to fix.

You want mistakes to grab you and force you to fix them now. You do not want them to get past you, falsely pass integration tests, and come across to QA, or end users, as a logic flaw in seemingly unrelated code.

Again, in the typical cover-up strategy, dependency-checking code becomes clutter and adds complexity. You may find yourself spending almost as much time debugging dependency checks in cases like this, as actually fixing functional code. Part of a well-designed exception strategy is writing clean code that keeps dependency checking to a minimum.

One way to make that possible is to set up encapsulation boundaries. This is also a good practice for other reasons, including managing security. Top level processes delegate all their activities so they can supervise them. Validate resources and lock them as close to the start of your process as possible, and throw when it fails. Validate data and its schema when and where it enters the process, and throw when it fails. Validate authentication & authorization as soon as you can, and throw when it fails. Once you have your dependencies sanity-checked, clean processing code can begin processing.

Don’t forget that the UI is an edge. Not only should input be validated, but due to important needs of the user, reconciliation code that respects the user should be set in place.

Thrown exceptions, of course, need to be of well-known domain exception types that can be easily identified in the problem domain, and well handled. Don’t go throwing framework exceptions. It will be too easy to confuse a framework exception for an application exception. Some frameworks offer an “ApplicationException” designed for the purpose. I might consider inheriting from ApplicationException. Throwing an “ApplicationException” is probably not unique or descriptive enough within the problem domain to make the code understandable.

By taking an encapsulation boundary approach, lower level tasks cleanly assume their dependencies are valid. If a lower level task fails due to a failed assumption, you’ll know instantly that it’s a bug. You’ll know the bug is either right there on the failure point, or it’s on the edge where the dependency or resource was acquired.

Another important consideration in an exception-handling strategy: code reusability. It often makes sense to decouple a supervisor from its process. In this way, it becomes possible to apply consistent supervision over many different processes. Another interesting possibility that becomes possible in this manner is the idea of code execution with design-time or even run-time configurable supervision. Each different “supervisor” construct could respond to the informational needs of a different set of stakeholders. Finally, handling can be decoupled from supervision. This provides another way to make stakeholder needs configurable.

Ok, so I'm getting carried away a bit there... but you get what I'm saying.

By designing encapsulation boundaries, and by decoupling the various parts of exception handling, you can support effective supervision and reconciliation through each phase of the project life cycle via configuration. In this way, you can honor all the stakeholders during their tenure with the application, and write better software.

[3/12/08 Edit: I originally liked the term "trust boundaries", because it focused on being able to trust dependencies, and also brought security as a dependency to mind, but "encapsulation boundary" is much more precise. Thanks, Kris!]