Solution: Users Don't Receive Email Sent by an Application

I don’t know much about Exchange, so was baffled when one of of one of our applications couldn’t send emails to an Exchange-based distribution list.

Symptoms

An application sends regular emails to numerous users, and this has been working for some time. A new Exchange based distribution list was set up so that a group of users could receive some of the emails.

  • Users in the distribution list receive the emails if the application was set to send them to their regular account.
  • If I send an email from my account, it reaches everyone in the distribution list.
  • Everyone else receives the mails from the application.
  • Nobody in the distribution list receives emails.

Diagnosis

The new distribution list is the only thing that has changes, so there was obviously something wrong with the list.
I asked one of our Exchange admins to investigate. Exchange message tracking suggested that the emails were being bounced back to the sending account.
Looking at the sender’s returned email revealed the problem with the distribution list.

Explanation

The distribution list was configured to only accept emails from authenticated users. Mails to the list that come from me get through because I’m logged in. However, the application doesn’t authenticate with Exchange when it sends emails. As a result they were being blocked by Exchange.

Solution

Change the settings on the distribution list so that it will receive emails from non-authenticated users.

Lessons Learned from a Failed Deployment

Last week we were scheduled to replace a critical component in a complex, mission-critical hospital system. About two-thirds of the way through the deployment, it became clear that I had missed something during the preparatory work for the change (security, always check security). Additional work would be needed before we could complete the upgrade, and it was very likely that we wouldn’t finish the deployment on time…

Lessons from Previous Implementation Projects

Given the critical nature of this change, experience told us that we needed to do things “properly”. Previous experience suggested that we needed to:

  1. Test the new solution thoroughly (we put 2,000,000 transactions through the new component and compared the results to the old solution).
  2. Write a sufficiently detailed implementation plan
    1. Include prep-work required prior to implementation
    2. Include enough detail so you don’t have to think during implementation. This helps under pressure, and ensures that energy is available to tackle the unexpected.
    3. Outline post-implementation work required
  3. Test the implementation plan (This was not possible for us due to differences between our test and live environments. Rectifying this would cost £ hundreds of thousands).
  4. Write a sufficiently detailed roll-back plan
  5. Test the roll-back plan (Again, not possible).
  6. Keep users and stakeholders informed… allowing plenty of time for them to make necessary arrangements for down-time.
  7. Define a change window
    1. When you’ll cause least disruption during the change
    2. When failure of the new component will cause least chaos
    3. When you have enough support from others
    4. When you’ll have enough time for post-implementation testing
  8. Get approval from stakeholders… in writing
    1. Explain the purpose of the thing you’re changing
    2. Explain why you’re making the change
    3. Say how things are at the moment
    4. Say how things will be in the future
    5. Explain how you will monitor the new solution
    6. Prepare your implementation and roll-back plans in advance
  9. Check the state of the system before changing it (so we could be sure that any faults were due to our changes and not existing faults)

Well, we had done all that, but I had made a minor mistake during prep, so things were going badly.
So, my team leader made the call: to roll back.

Lesson 1: Be Prepared to Roll Back

I don’t just mean having a written plan, although that was extremely useful. I mean psychologically. It is sometimes hard to admit defeat. However, it is better to roll-back than either (1) upset customers by breaching the change window and (2) making mistakes whilst working under pressure. It just isn’t worth it.

Lesson 2: A Successful Roll-Back Is Not a Failure

… it is a tactical retreat. As we had a good roll-back plan we were able to revert to the old module without loss of data, and to do so within the change window. We had maintained the status quo.

Lesson 3: Roll Back Completely

You really don’t want to leave a system in an indeterminate state. As it was, we left some things in place ready for our next roll-out attempt. This was a mistake, as it caused (1) some minor confusion, and (2) if we had forgotten and done more testing, we could have caused corruption of live data.

Lesson 4: Communicate

We explained the reasons for the roll-back and the steps we had taken to make sure that we wouldn’t experience the same difficulties again. This was trust-building, and others were supportive of our action.

Lesson 5: Rally and Retry

Once the roll-back was verified the others went home. I stayed late to fix the problem that had caused the implementation issues.

Conclusion

This was a great learning experience for me. Today we did the implementation again. But this time, it went smoothly.

Reset Microsoft Word

I recently had problems with Word style getting messed up, so I figured I’d just zap the Normal.dot file and everything would be OK. It wasn’t, so I took more drastic measures.
It turns out you can use the registry editor to blow away Word’s settings, and Word will restore the defaults when you start it next. The key that holds the settings varies according to the version of Word that you’re running.
Warning: If you think this is a safe thing to do, don’t do it.

Word 97 HKEY_CURRENT_USER/Software/Microsoft/Office/8.0/Word
Word 2000 HKEY_CURRENT_USER/Software/Microsoft/Office/9.0/Word
Word 2002 HKEY_CURRENT_USER/Software/Microsoft/Office/10.0/Word
Word 2003 HKEY_CURRENT_USER/Software/Microsoft/Office/11.0/Word
Word 2007 HKEY_CURRENT_USER/Software/Microsoft/Office/12.0/Word

Metaphors for Programming

In the opening chapter of “The Developer’s Code“, author Ka Wai Cheung describes the book as “a collection of lessons, observations, and missteps I’ve gathered, first-hand, in our industry”.

The following is my summary of Chapter 2: Metaphor. Whilst I’ve tried to do justice to the author’s intentions, the interpretation is my own. I’ve included my comments in italics.

Follow metaphors with care

In other professions (cooking, music) people’s experience tells them if something is good or bad. However, programmers must resort to metaphors like “software architect” because people have little experience with code. Whilst metaphors are useful, they can distort our priorities if relied upon too heavily.

Several examples follow:

Plan enough then build

The architecture model over-emphasises planning over writing code and testing it. Agile development unbinds us from the metaphor.
In construction, the product requires detailed specifications:

  1. re-work is expensive or impossible
  2. task scheduling is complex
  3. raw materials are limited
  4. errors during production are expensive

Whereas in software production:

  1. the construction remains malleable
  2. the product is testable in (near) real situations
  3. errors during production needn’t equate to issues in use
  4. the product can be duplicated at no cost

Launch is just the first release

Buildings are built and then opened. In contrast, software can mature and develop over time, and users are increasingly accustomed to change.
Launch is a milestone, but not the end of a software products life-cycle. Get the important stuff right and fix the small stuff over time.

The “ivory tower” architect is a myth

Real architects don’t hammer in nails, but the division between designing and building software is arbitrary. In reality, software architects need the insights that are only available to coders. Software architects must, therefore, find time to do at least some coding.

Throw away your old code

Unlike real-world materials that are often reused, we don’t need to keep old code because:

  • Unlike real-world materials (bricks, metal and so on) the raw elements of code are inexpensive and easily replaced
  • Commented-out code is distracting
  • We rarely uncomment code that we comment out
  • Resurrecting old code is often more expensive than rewriting it

An important exception to this rule is where you want to keep a record of your thinking rather than the code itself.

Diversification over specialization

Specialised roles in building occur in different physical places, whereas our tasks are all on the same screen. For this reason, there is no reason we can’t be good at a range of jobs.

The logic of this argument is flawed, and the conclusion questionable. While over-specialising can be dangerous, trying to be a jack-of-all-trades can be equally hazardous.

My guess is that this is just a weak example of a powerful idea: the author is trying to illustrate the idea of architecture as a limiting metaphor rather than provide a justification for becoming a generalist.

Metaphors Hide Better Ways of Working

Metaphors both liberate and constrain our thinking.
Where they are a problem:

  1. Learn to see where metaphors are being used and their limitations
  2. Try changing metaphors

HTML CSS and Real Programming Languages

It is a universal rule:
Wherever people talk about web programming languages:

  1. Someone will mention HTML and CSS
  2. Someone else will protest that HTML and CSS are not programming languages

But, does it matter?

A True Story

I once spent a few months developing a complex performance management system using a “proper” programming language. It automated the process of gathering data from various other systems, performed various manipulations on that data and then presented the results in a simple table, a kind of balanced scorecard.
Once it was live, a senior manager saw a tool written for a similar business, and asked if we could replace what I had written using their alternative. Furthermore, he was upset that I’d spent weeks developing my solution, where this new app had been written in a matter of days.
Further investigation revealed that the manager’s preferred option was all presentation with no substance: there were lots of colours and icons and graphs, but there was absolutely no logic behind them. All the data had to be gathered and manipulated manually. Despite the pretty interface, the application was essentially useless.
I’m happy to say that the manager in question was persuaded that my approach was the one that met his real business needs.

The Importance of Real Programming Languages

Now, as any CS graduate will know,  “real” programming languages are Turing Complete. As a proper sad geek, I find this sublimely fascinating. It means that, for any program written in a T-C language, a functionally equivalent program can be written in any other T-C language. Of course, this isn’t to say that all languages are the same. The each have their strengths and weaknesses that make them more or less suitable for certain tasks. However, I/O aside, this means that all programs can be written in all true programming languages.
Of course, the same isn’t true of a markup language like HTML or CSS. In fact, there are whole classes of problem that these languages simply can’t solve. Where I can program anything I want in a true programming language – including layout engines – it just isn’t possible to achieve the same things with languages that aren’t T-C.
(As an aside, some people think that the fact that HTLM and CSS are declarative rather than imperative actually nakes a difference here. But, that really isn’t the point. It is possible for a declarative language to be T-C and for an imperative language to be not T-C)

The Importance of Presentation

Unfortunately, when programmers get sniffy about HTML and its ilk not being true programming languages, there is often an implication that skills in HTML, CSS etc. are somehow inferior to skills in “real” programming languages. This is a serious mistake.
In my story, the senior manager felt that design was very important to him, to the extent that he was initially prepared to overlook function in its favour. Now, if this were an isolated incident, I might suggest that the manager was just being foolish.
But it wasn’t.
Time and again, I’ve met users who are impressed by flashy graphics and whizzy widgets, but unimpressed by raw functionality and my technical achievements. I think that there are several lessons to learn here:

  • People evaluate software on criteria that they understand. They often understand the difference between good-looking and ugly, but rarely appreciate technical nuances.
  • People are fooled by appearances. This may not be a good thing, but it is a reality that we must live with.
  • Appearances influence the way people feel about software. The way people feel about software is important to them. Indeed, people sometimes prefer software that makes them feel good over software that is functionally superior. Indeed, they might well be more productive with feel-good tools than with technically superior tools. To this extent, our users are not being fooled. They are actually making a wise and thoughtful choice.
  • As programmers, we often neglect the role of presentation as we focus on function. To some extent, this is right and proper. However, it is important to recognize that there is another dimension to our work that is important to our customers.

So, presentation-oriented languages (HTML, CSS) are important. The value added by tose who can use these tools effectively should not be underestimated.

What is the Difference, Really?

The interesting thing is that, to most people, none of this T-C business matters. The important difference between a programming language and HTLM / CSS is this:

HTML and CSS describe presentation, whereas programming languages describe function.

And both of these things matter.

Why are Programmers Pedantic About it All?

So, why do programmers spend so much passion on defending their position that HTML and CSS are not programming languages? There are lots of reasons.

  1. Because there really is a real and fundamental difference, as outlined above.
  2. Programmers spend a great deal of time, effort and money developing their skills. People naturally value the things in which they invest (“your heart is where your money is”).
  3. Programmers often feel the need to justify the amount of time it takes to produce results compared to the rapid results achieved by UI designers. In order to do this, they need to draw a distinction between what the two groups actually do.
  4. Employers need to apply the right people to the right jobs. Unless we clarify the (often technical) differences, managers easily make the wrong calls.

Is it Always Appropriate to be Pedantic?

Let’s face it, as programmers we’re a naturally pedantic lot. It goes with the territory. It doesn’t help that many of us have been burned when non-programmers have failed to understand what we do.
Nevertheless (and to be honest, this goes against my natural instincts), I don’t think we need to call people out whenever they slip over every little distinction.

The important things here are context and perspective.

I’m told that, to a biologist, a tomato is a fruit. But when I buy them in the supermarket, I look for them amongst the veg. Why? Because the technical distinction doesn’t matter in that particular context. Moreover, the distinction would actually get in the way if I daft enough to include tomatoes in a fruit salad.
It is the same with computer languages. There are times when the difference between programming languages and other languages really does matter. Quite often, however, we can all communicate perfectly effectively when just lump them all in together. In the case when people are asking what programming languages they need to lear to do web development, it really didn’t matter what languages were true programming languages and which were not. Pointing out the distinction won’t advance the discussion in any way. Indeed, in many cases, pedantry can stir up negative feelings and damages relationships… as my wife will gladly testify!

When to Make a Stand?

A preacher friend of mine once delivered a sermon entitled:

Is this a hill worth dying to be on?

He was referring to generals who make a strategic assessment over which battles are which fighting: are the gains worth the costs?

  • Is it really worth interrupting the flow of the discussion to make this distinction?
  • Does my pedantry stem from a sense of arrogance or from past hurt?
  • Do my comments value the skills of others as well a my own?

Of course, there are times when distinctions need to be made. My aim is that, when I make a contribution, it will add value to our collective endeavours.
That is, after all, the job of a every real programmer.
(This article is based on my answer to a question on Programmers)

Because there really is a real and fundamental difference, as outlined above.

What Makes Technology Popular?

It is easy to assume that technology becomes popular simply because it is better than the competition. But as the saying goes, “that ain’t necessarily so”. In fact, the history of innovation is replete with examples of inferior

technologies capturing a larger market share than their technically superior rivals.

Some of the best known (and hotly debated) examples of include:

  1. Betamax / VHS
  2. Ethernet / Token Ring
  3. Mac / Windows
  4. Firefox / Internet Explorer

Moreover, there are also plenty of perfectly good technologies that die a death because they lag far behind their rivals in terms of sales, despite their being little difference in their technical merit. Most format wars are a case in point.

So, why does this happen? How does a technology come to dominate the market, often at the expense of other, sometimes superior, alternatives?

The answer lies in the way that individuals make decisions about the technology they use. The technology that gets picked for three reasons:

  1. It meets a need (or at least appears to)
  2. It gets publicity – sometimes through deliberate advocacy, sometimes through word of mouth
  3. There costs of adoption are outweighed by the benefits

To illustrate, we’ll consider a well-known example: the Windows operating system.

Firstly, users who get what they need from Windows are unlikely to go looking for alternatives. They might do so, however, if they start to believe that their are problems with Windows that reduce their productivity. For the majority of Windows users, however, Windows meets their needs. Whether that is because it is a great product or because the users don’t know any different is irrelevant. The point is that for many users, it is simply good enough.

Secondly, even if they go looking, Windows users are unlikely to find out about many of the alternatives. This is because they just don’t receive very much attention from developers, other users nor the press. How many Windows users have heard of the Haiku Operating System, for example? On the other hand, it is more likely that they will have heard of the Mac or Linux.

Finally, even if Windows users find an alternative, they won’t switch unless the benefits of switching outweigh the costs of adopting the new platform. If critical applications are available on the new platform, users might switch. If not, they are unlikely to do so. Similarly, if the learning curve is steep, they may be reluctant, but if the user experience is similar, they may be more willing to do so. Ubuntu is a popular Linux distribution because of the focus on usability, whereas others are less so because they’re difficult to get used to.

So, there you have it: technology is not popular just because it is good (although that helps), but because of market forces; marketing in particular.

Thanks to the jgauffin for the inspiration behind this post.

Windows Textbox Shortcuts

Form Navigation

Tab Move to the next input area on a form
Shift + Tab Moves to the previous input area on a form

Moving the Caret

Right Arrow Moves the caret one character to the right
Left Arrow Moves the caret one character to the left
Ctrl + Right Arrow Moves the caret to the start of the next word
Ctrl + Left Arrow Moves the caret to the start of the previous word
Up Arrow Moves the caret up one line
Down Arrow Moves the caret down one line
Ctrl + Up Arrow Scrolls the text up one line
Ctrl + Down Arrow Scrolls the text down one line
Home Moves the caret to the start of the current line
End Moves the caret to the end of the current line
Ctrl + Home Move the caret to the beginning of the text
Ctrl + End Move the cursor to end of the text

Selecting Text

Ctrl + A Selects all text
Shift + Right Arrow Extends the selection one character to the right
Shift + Left Arrow Extends the selection one character to the left
Shift + Up Arrow Extends the selection up one line
Shift + Down Arrow Extends the selection down one line
Shift + Home Extends the selection to beginning of line
Shift + End Extends the selection to end of line
Shift + Ctrl + Right Arrow Extends the selection to the next word break
Shift + Ctrl + Left Arrow Extends the selection to the previous word break
Shift + Ctrl + Up Arrow Extends the selection to the paragraph above
Shift + Ctrl + Down Arrow Extends the selection to the paragraph below
Shift + Ctrl + Home Extends the selection to the top of the text
Shift + Ctrl + End Extends the selection to the bottom of the text

Cut, Paste, and Undo

Ctrl + X, or
Shift + Del
Cuts the selected text
Ctrl + C, or
Ctrl + Ins
Copies the selected text
Ctrl + V, or
Shift + Ins
Pastes the selected text
Ctrl + Z Undoes the last edit
Ctrl + Y Redoes the last edit

Deleting Text

Backspace Deletes the previous character
Del Deletes forward to the next word break
Ctrl + Backspace Deletes back to the previous word break
Ctrl + Del Deletes forward to the next word break
Ctrl + Alt + Del Kaboom!

Rich Text

Ctrl + B Bold
Ctrl + I Italic
Ctrl + U Underline

Some Basic Interview Advice

Notes from a recent training session (rather a basic one – but fundamentals are easily forgotten in stressful situations) :

Advance Preparation

  • Keep a copy of the job advert and specification.
  • Keep a copy of your application form / CV
  • Research the department / organisation + make notes
    • Understand where the organisation is going
  • Prepare questions to ask the interviewers

On-the-Day Preparation

  • Re-read your key documents – your application form / CV etc.

General Tips

Be:

  • Punctual
  • Smart
  • Polite
  • Natural but not casual

During the Interview

  • Take your time
  • Ask for repetition / clarification of questions – either because you need them or to give you time to answer
  • Give brief answers – ask if more detail is required.
  • Answer questions

IT Security: Some Notes

A few notes on the basics of IT Security:

Introduction to Security

Security is concerned with preserving the value of resources.

Valuing Data

The value of data is determined by its contribution to the goals of the one who owns that data. This is the same as the costs to the enterprise of loosing that data
The cost of security must be weighed risk of that data being compromised.

risk = (likelihood of data loss) x (cost of loss of data)

Security and Value

Security consists of:

  1. Identifying a resource that has value.
  2. Maintaining that resource.
  3. Preventing misuse of that resource.

The identification of value is a business decision.

Maintaining Resources

Integrity of Data

Maintaining data integrity is the process of ensuring that data is in a state which is consistent with expectations, the domain modelled by the system (accuracy), other data and itself. Data integrity is primarily concerned with software measures and procedural measures.
Examples of measures:

  1. Data validation.
  2. Making sure that users know if changes to the data are permanent or temporary.
  3. Protecting data from concurrency issues.
  4. Atomic transactions.

Safety of Data

The main method for keeping data safe is reliable storage of that data. Reliability is enhanced by redundancy. A backup is a relatively cheap, easy way to ensure that data can be recovered to a specified state following loss or corruption of that data.
Backups can be full or incremental.
The rules of a worthwhile backup:

  1. Have a backup
  2. Be able to identify the backup
  3. Be able to restore the backup
  4. Keep the backup secure

Of course, a backup will not be used if users are unaware of data corruption or loss.

Safety of Systems

Disaster recovery is the restoration of a system (rather than just the data) following the system’s failure.
Requirements:

  1. A backup of data
  2. Duplicate hardware / software environment
  3. Defined and tested procedures for recreating the lost environment

Preventing Misuse of Data

Authentication

Actions on data are initiated by an agent. Validity of an action is often determined by the identity of an agent. The identification of an agent is often termed authentication.
In order to verify the identity of the agent, the system can request that the information provide information about:

  • Something the agent knows: e.g. a password.
  • Something the agent has: e.g. an identity card.
  • Something the agent is: biometrics.

Authorisation

Authorisation is the process of allowing / disallowing an authenticated agent (often a user) to perform a certain action.
The action may be as simple as allowing the agent to read the data:

  • Privacy: ensuring that only authorised agents can read the data.
  • Access: ensuring that all authorised  agents can read the data.

Enforcing Complicity

An often overlooked mechanism that could be used to ensure only appropriate actions are taken on data is to enforce rules that require two or more trusted individuals to authorise action on a resource.
For example, a safe that can only be opened if both key-holders are present.
One possibility rarely implemented is to have an operator initiate a request for change that must be approved by a business manager who has has no ability to initiate the request herself.