Troubleshooting

These general steps can be used to troubleshoot anything.

Step 1: Ask Questions to Understand the Presenting Problem

  • Who is effected?
  • What happens?
  • What should happen?
  • Has it ever worked?
  • Where does it happen?
  • Does it happen for everyone?
  • When does it happen?
  • What were you doing?
  • Has anything changed?

Step 2: Gather Evidence

Seek out information that will help you investigate and to support your understanding.
From, for example:

  • Other reports
  • Logs
  • Attempts to reproduce the issue

Step 3: Formulate a Theory

Step 4: Test to Disprove

It is important to attempt to disprove to overcome “Confirmation Bias”.

Step 5: Provide Recommendations for Further Investigation

Based on understanding of impact vs cause.

Sources of Data on IT Operations Performance

Data that tells you what is going on in your IT infrastructure comes from the following sources:

  • Wire Data (network packets)
  • Machine Data (e.g. CPU usage for workstations and servers)
  • Agent Data (that tell you current number of users, which applications are used when etc.)
  • Synthetic Data (data that you check as required, e.g. “is it available now”)
  • Human Generated Data (e.g. knowledge bases)

It is essential to gather data, so decisions are made based on information rather than guesses.

Solved: Missing Target Framwork in Visual Studio

Here is how I fixed my missing .Net target framework 4.0 issue in VS2010.

The Symptoms

Visual Studio 2010 stopped detecting the .Net framework 4.0. Specifically, this framework version was missing from the “Target framework” drop-down on a project’s property page. Versions 2.0, 3.0 and 3.5 were present in the list, but not 4.0. Reinstalling the framework and rebooting had no effect.

The Solution

The problem was that the file,”FrameworkList.xml” was missing from folder “C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\RedistList“. Copying this file from another machine solved the problem.

I am now able to target .Net 4.0 again:

Further Reading

  • I found this solution at the end of this discussion.
  • There is also a stackoverflow question dedicated to this issue.

Solved: BizTalk Map Renaming Fails in Deployed Application

BizTalk map renaming isn’t quite as simple as renaming the map.

The Problem

I renamed a BizTalk map in VisualStudio’s Solution Explorer, compiled and deployed the application. However, in the BizTalk Management Console, the renamed map still shows up with its old name.

The Solution

This is one of those BizTalk things that we’ve all been caught out by at least once.
The solution is to right-click the map name in Solution Explorer and click properties. In the properties window, change the name of the Type to the name of the map. Then redeploy.
BizTalk Map Renaming

Everything is Suddenly Asking for Admin Permissions: Solved

The Problem

All of a sudden, odd things have stopped working and others are asking for admin permissions. Explorer won’t let me see things I can normally see, scripts aren’t running… This seems odd, as I’m already an admin on my box. What’s going on here?

The Conditions

This has happened to me several times now, and each time the circumstances have been the same:

  • I’m Windows on a network
  • My password has changed recently
  • I haven’t logged in recently on the box where I’m having problems

This is quite a common scenario for me at work because most of my programming work is done on a VM that I very rarely restart, or even log off. At the same time, whenever my password expires I change it on my physical box rather than the VM.

The Solution

Log out and back in again.

The Reason

Your credentials for your current session are stale.

Solved: BizTalk Scripting Functoid Inline Script Issue

The Problem

I have been developing a rather complex map that includes various scripting functoids for manipulating dates.  One of the in-line C# scripts started producing output that simply didn’t make sense. I ran the code in LINQPad, and it produces the expected output, but testing the map resulted in some bizarre behaviour.

My code looked like this:

Given an input node that contains:

1936-08-07T00:00:00

I expected:

19360807

But received:

193608071200

The Solution

I validated the map, which generated the XSLT that is actually run on the input. I was surprised to find that the C# code embedded in the XSLT looked like this (notice the difference in the string formatting on the 6th line):

Why is this different from the code in the scripting functoid?

MSDN documents the answer:

Avoid using the same method signature more than once. When several Scripting functoids have the same method signature, BizTalk selects the first implementation and disregards the others.

It turned out that I’d created a similar functoid elsewhere in the map that uses the same method signature (name and parameters), but had the implementation above. It turns out that BizTalk recognized that more than one function was defined with the same name, and then silently ignored all but the first one.

Further Reading

Solution: Users Don't Receive Email Sent by an Application

I don’t know much about Exchange, so was baffled when one of of one of our applications couldn’t send emails to an Exchange-based distribution list.

Symptoms

An application sends regular emails to numerous users, and this has been working for some time. A new Exchange based distribution list was set up so that a group of users could receive some of the emails.

  • Users in the distribution list receive the emails if the application was set to send them to their regular account.
  • If I send an email from my account, it reaches everyone in the distribution list.
  • Everyone else receives the mails from the application.
  • Nobody in the distribution list receives emails.

Diagnosis

The new distribution list is the only thing that has changes, so there was obviously something wrong with the list.
I asked one of our Exchange admins to investigate. Exchange message tracking suggested that the emails were being bounced back to the sending account.
Looking at the sender’s returned email revealed the problem with the distribution list.

Explanation

The distribution list was configured to only accept emails from authenticated users. Mails to the list that come from me get through because I’m logged in. However, the application doesn’t authenticate with Exchange when it sends emails. As a result they were being blocked by Exchange.

Solution

Change the settings on the distribution list so that it will receive emails from non-authenticated users.

Lessons Learned from a Failed Deployment

Last week we were scheduled to replace a critical component in a complex, mission-critical hospital system. About two-thirds of the way through the deployment, it became clear that I had missed something during the preparatory work for the change (security, always check security). Additional work would be needed before we could complete the upgrade, and it was very likely that we wouldn’t finish the deployment on time…

Lessons from Previous Implementation Projects

Given the critical nature of this change, experience told us that we needed to do things “properly”. Previous experience suggested that we needed to:

  1. Test the new solution thoroughly (we put 2,000,000 transactions through the new component and compared the results to the old solution).
  2. Write a sufficiently detailed implementation plan
    1. Include prep-work required prior to implementation
    2. Include enough detail so you don’t have to think during implementation. This helps under pressure, and ensures that energy is available to tackle the unexpected.
    3. Outline post-implementation work required
  3. Test the implementation plan (This was not possible for us due to differences between our test and live environments. Rectifying this would cost £ hundreds of thousands).
  4. Write a sufficiently detailed roll-back plan
  5. Test the roll-back plan (Again, not possible).
  6. Keep users and stakeholders informed… allowing plenty of time for them to make necessary arrangements for down-time.
  7. Define a change window
    1. When you’ll cause least disruption during the change
    2. When failure of the new component will cause least chaos
    3. When you have enough support from others
    4. When you’ll have enough time for post-implementation testing
  8. Get approval from stakeholders… in writing
    1. Explain the purpose of the thing you’re changing
    2. Explain why you’re making the change
    3. Say how things are at the moment
    4. Say how things will be in the future
    5. Explain how you will monitor the new solution
    6. Prepare your implementation and roll-back plans in advance
  9. Check the state of the system before changing it (so we could be sure that any faults were due to our changes and not existing faults)

Well, we had done all that, but I had made a minor mistake during prep, so things were going badly.
So, my team leader made the call: to roll back.

Lesson 1: Be Prepared to Roll Back

I don’t just mean having a written plan, although that was extremely useful. I mean psychologically. It is sometimes hard to admit defeat. However, it is better to roll-back than either (1) upset customers by breaching the change window and (2) making mistakes whilst working under pressure. It just isn’t worth it.

Lesson 2: A Successful Roll-Back Is Not a Failure

… it is a tactical retreat. As we had a good roll-back plan we were able to revert to the old module without loss of data, and to do so within the change window. We had maintained the status quo.

Lesson 3: Roll Back Completely

You really don’t want to leave a system in an indeterminate state. As it was, we left some things in place ready for our next roll-out attempt. This was a mistake, as it caused (1) some minor confusion, and (2) if we had forgotten and done more testing, we could have caused corruption of live data.

Lesson 4: Communicate

We explained the reasons for the roll-back and the steps we had taken to make sure that we wouldn’t experience the same difficulties again. This was trust-building, and others were supportive of our action.

Lesson 5: Rally and Retry

Once the roll-back was verified the others went home. I stayed late to fix the problem that had caused the implementation issues.

Conclusion

This was a great learning experience for me. Today we did the implementation again. But this time, it went smoothly.

Calling Constructors in C# Using Reflection

Reflection can be used in C# to invoke a class’ constructor. This is useful when you don’t know the type of the class that you want to instantiate until runtime.
As usual, there are various options open to you depending on your requirements.

Invoking the Default (Parameterless) Constructor

When You Can Pass the Required Type as a Generic Parameter

Under these circumstances, you don’t need to use reflection at all:
[code language=”C#”]
T Instantiate() where T : new()
{
return new T();
}
[/code]
Alternatively, you can call the generic version of Activator.CreateInstance:
[code language=”C#”]
ObjectType instance = Activator.CreateInstance();
[/code]

When You Know the Required Type

You can instantiate using either the type itself:
[code language=”C#”]
ObjectType instance = (ObjectType)Activator.CreateInstance(objectType);
[/code]
Or the type’s name:
[code language=”C#”]
ObjectType instance = (ObjectType)Activator.CreateInstance(“MyNamespace.ObjectType, MyAssembly”);
[/code]

When You Need a Reference to the Default Constructor

This is useful if you want to call the constructor later.
[code language=”C#”]
var constructor = type.GetConstructor(Type.EmptyTypes);

ObjectType instance = (ObjectType)constructor.Invoke(null);
[/code]

Invoking a Parameterized Constructor

If you want to get a reference to a constructor that has, for example, two parameters, the first of which is a long and the second an int:
[code language=”C#”]
// We need the constructor with the following signature:
var argTypes = new[]
{
typeof (long),
typeof (int)
};
ConstructorInfo constructor = ObjectType.GetConstructor(argTypes);
[/code]
This constructor can then be invoked like this:
[code language=”C#”]
var argValues = new object[] { lngFirstParameter, intSecondParameter };
ObjectType instance = (ObjectType) constructor .Invoke(argValues);
[/code]

Getting Information About Constructor Parameters

The following method will write out the parameters of type ObjectType:
[code language=”C#”]
public void PrintParameters()
{
var ctors = typeof(ObjectType).GetConstructors();
// Assuming class ObjectType has only one constructor:
var ctor = ctors[0];
foreach (var param in ctor.GetParameters())
{
Console.WriteLine(string.Format(“Parameter {0} is named {1} and is of type {2}”,param.Position, param.Name, param.ParameterType));
}
}
[/code]

References