This is a thread on the TimeBomb 2000 bulletin board. I plan to continue with this debate, and will post the results here as they are completed.
No one seriously questions the fact that IT systems operate at less than 100% today. Errors in IT systems are a fact of life. And that the "system", if you will, has not collapsed to date from IT errors.
From this, it would appear that the "burden of proof" would rest on those claimimg that Y2k will collapse the system. Past experience shows that this sort of collapse has not happened to date.
It has been postulated, however, that Y2k is a singular event. That Y2k is "systemic", and has the potential to generate massive, simultaneous errors, across IT systems. That past experience is no predictor in this case, since it is a singular event.
So, we have do doubt that global IT systems have a built-in "fault tolerance". The question is, will the Y2k rollover exceed this "fault tolerance"?
While it may be virtually impossible to estimate fault tolerance level, it is my contention that IT systems in general have, almost without question, in the last 8 months experienced errors orders of magnitude greater than that which will be experienced at the rollover to 2000. And that in all liklihood, we have already experienced several "singular events" relating to Y2k errors, that are at least of the same order of magnitude of the Y2k rollover. And that little or no effect of these has been felt by the average person. And certainly, no collapse has taken place.
To estimate the actual error rates, the following assumptions and sources will be used:
1) Starting point is the universe of non-compliant systems at the start of 1998. These are the systems at risk for generating Y2k failures. The base unit of measure for a failure is a "function point". For those interested, see Capers Jones' article What Are Function Points? for a detailed definition. But in a nutshell, a function point is an Input, Output, Inquiry, Data File, or Interface in an application. Function points are then derived by weighting each of these.
2) Metrics for error rates, date logic, etc., are taken from Capers Jones' article Probabilities of Year 2000 Damages. Jones is probably the foremost authority and source on software metrics, and is quoted by many, including Ed Yourdon in his analysis.
3) The "singular event" that has the most potential to cause simultaneous errors, and as such the greatest risk of collapse, is the two-week period surrounding Jan 1, 2000. Gartner Group has estimated that 10% of potential Y2k errors will occur during this time frame (source: Slide 2). To add a level of certainty, this analysis will double the Gartner Group estimate, and assume 20% of Y2k errors will occur during this time frame.
4) An assumption is made that 66% of non-compliant systems will be either remediated or replaced, leaving 33% untouched. This assumption I would characterize as somewhat middle of the road. Capers Jones has estimated a "Best Case" scenario of 85% in the US, 75% in Europe, and 65% elsewhere (source). Other surveys, such as an often quoted Cap Gemini survey of US firms (source), cite 78% expecting to have more than 76% of their code fully fixed and tested.
5) An assumption that, of the non-compliant systems that are addressed, 50% were replaced, and 50% remediated. Based on personal experience, the percentage replaced would be much higher, but I recognize the bias within my SAP experience. A quick review of the Fortune 500 level SEC statements will find, for example, SAP mentioned in a large number of cases. And, as the business size decreases, the liklihood of the business using canned applications, as opposed to custom-developed systems, increases. As well, I think the failure of the predicted COBOL programmer shortage to materialize is also an off-shoot of the fact so many systems were replaced.
6) Finally, that system implementations typically occur month-end intervals. Particularly true of financial systems, but in general as well. This again is based on my experience with system implementations.
With the above sources and assumptions, I will first make an estimate of the percentage of function points that will fail during the Y2k rollover.
Capers Jones estimates that, on average, 15% of the function points within an application contain Year 2000 date errors. Our assumption is that 33% of these are untouched, and remain within the code. This leaves 5% of the function points with untouched Y2k errors.
Jones also estimates that in general, IT is only 95% efficient at finding and fixing Y2k errors. 10% of the function points are either fixed or replaced, and our assumption is that 50% of those are fixed. So 5% of the function points have gone thru remediation. With the 95% efficiency, it leaves .25% of the function points that have been remediated still containing Y2k errors. Adding together, 5.25% of function points will still contain Y2k errors.
But these errors are spread out. The doubling of Gartner Group's estimate gave 20% of errors occuring during the rollover. Applying this gives 1.05% of function points failing during the rollover.
So, our baseline is 1.05% of function points failing on rollover.
Is this a singular event, with an unprecedented number of errors? I truly do not think so.
Our universe of errors begins at the start of 1998. We have 24 month-end periods until the year 2000, where systems implementations occur.
Start by distributing the unremediated or missed Y2k errors. Gartner Group estimates 25% of the errors will occur in 1999. So 80% of the 5.25% of function points with Y2k errors will not occur at rollover, or 4.2%. 25% of these will occur during 1999, or 1.05%. Spreading these out over the 12 periods in 1999 gives .0875% of function points failing due to Y2k errors at each period.
Next, we need to deal with errors introduced through remediation. Capers Jones calls these "Bad Fixes". In essence, any time code is modified, errors can and will be introduced.
Jones estimates that in fixing function points, 10% will introduce new errors, 70% of which will be caught. So 5% of the function points are being fixed. 10%, or .5%, will contain new errors, of which 70% will be caught. So .15% of the function points will contain new errors. These are not spread out according to Y2k error distributions, but can occur literally anywhere. The vast majority of remediated applications are being reimplemented in 1999. So again, spreading this out over the 12 periods in 1999 leaves .0125% of function points failing due to new errors introduced due to remediation at each period.
Adding this to the .0875% of function points failing because of unremediated Y2k errors gives .1% of function points failing at each period in 1999.
Finally, we need to consider errors due to system replacement. Implementations introduce errors at a much higher rate than normal remediation. Jones estimates that 5 errors per function point are introduced through new development. To be fair, not all errors are equal, just as many types of Y2k errors can be lived with. But to be ultra-conservative, let's assume only 15% of delivered errors in software implementations are comparable to Y2k errors. That leaves 75% of the function points in replacement systems containing errors on par with Y2k errors.
In our universe of non-compliant systems, only 33% were left untouched, leaving 66%. 50% of these were replaced, as opposed to remediated, leaving 33% of function points in non-compliant applications being replaced. From above, 75% of these will contain errors on par with Y2k errors, or 24.75% of function points.
These errors are spread out over the 24 periods in 1998 and 1999. I'll assume a uniform distribution, though again it probably should be weighted more heavily to this year. But uniform distribution leaves 1.03% of function points generating errors on par with Y2k errors at each of the 24 periods in 1998 and 1999.
Adding the previous error rate of .1% gives 1.13% of function points generating errors during each of the 12 monthends in 1999. This rate compares with the estimated baseline of 1.05% of function points generating errors at rollover.
Now, the point is not some false level of precision, but to estimate a general error level. And the metrics and estimates show that yes, in all liklihood each of the 8 month-end periods in 1999 has generated errors of the same level of magnitude that can be expected at the rollover.
Thus far, IT departments have dealt with the errors. No collapse has occurred. And I don't expect that to change at the rollover to the year 2000.
-- Hoffmeister (hoff_meister@my-deja.com), August 11, 1999
I think the basic error in this analysis is very simple. There is no reason whatsoever to believe the claims of organizations as to their "expected" rate of remediation of their systems. The reason I say this is that, both by personal experience as well as by studies of the likelihood of projects being completed on time, any claims of future completion of projects must be discounted very heavily. Remember, many organizations were planning to be finished by December 31st, 1998, with a full year for testing, and yet almost no one has announced that they made that deadline. So the studies that you cite, indicating the numbers of companies that "expect" to be finished with a specific percentage of their work, presumably by December 31st, 1999, simply point out that these organizations are not yet finished. Why should we believe them this time?
-- Steve Heller (stheller@koyote.com), August 11, 1999.
Actually, the analysis is based in no way on any single company completing their Y2k project. It is based on a percentage of function points that have been addressed, either through replacement or remediation.It is undeniable that some percentage of function points have been addressed. For example, merely the 20,000+ installations of SAP illustrate that fact. And, unless you are contending that corporations are out and out lying about the status of Y2k remediation, and that they are actually spending money on nothing, it is also undeniable that remediation is actually taking place.
The argument seems then to be the percentage. I made an estimate, using some fairly pessimistic sources. I could have, for example, cited the recently mentioned IDG survey 1000 companies here , where less than 2% expect to miss the the Jan 1 deadline. I did not, to allow a fairly large margin of error.
But, since you bring up studies on projects being completed on time, let's take a look. Ed Yourdon references Capers Jones statistics on projects being late in this article. I'll completely disregard the difference between software development projects, and maintenance projects, in order to provide another margin of error. (This is a distinction, by the way, that does not escape Jones).
The statistics show that, on average, 62.47% of projects are either delivered early or on time. 13.71% are late, and 23.82% are cancelled.
Now, it's doubtful many Y2k projects are merely cancelled. So lets lump that into late, and use 37.53% as late.
Off the bat, the 62.47% is very close to the 66% I used, and in fact would alter the analysis very little, and would not change the conclusion at all. But let's look further.
You mention the 12/31/98 deadline. Again, I have doubts about the meaning here; it does not seems to me most that made this claim were speaking of code renovation, as the "year for testing" implies they were not expecting to be complete then.
But taking it at face value, and using the study of software completions, it would appear that some large percentage would have, in fact, completed on time. The study suggests 62.47%. The study also states that, on average, projects that are late are late by 7.65 months. Which means that, using the study, half of those late would be completed about now, or another 18.765%, leaving a total of 81.235%. This percentage could then be extended to account for those completing later in the year.
So, even using the studies you reference, I could make a very strong case for a much higher percentage, over 80%, than the 66% I used.
-- Hoffmeister (hoff_meister@my-deja.com), August 12, 1999.
I think you've missed a very important change in Cap Gemini's estimates of the Y2K situation. It's true that according to your citation, from May 17th:Seventy-four percent of major corporations had expected to have more than half of their code "completely tested and compliant" by January 1 of this year, according to the last quarterly survey issued in December. But the current tracking poll reveals that only 55 percent actually reached this goal.
But according to their most recent news, as of August 10, 1999:
Fewer than half of America's largest companies (48 percent) expect all of their critical systems to be prepared for the Year 2000, according to a new survey by Cap Gemini America, Inc., an information technology and management consulting leader. One in five companies (18 percent) expect that 75 percent or less of their critical systems will be "completely tested and compliant" by December 31, 1999. Thirty-six percent expect between 76 and 99 percent of their applications to be ready for Year 2000, and two percent anticipate completing work on 50 percent or less of their systems. [emphasis added]
Please note that although the earlier survey said that 55% were "done" as of January 1st, the later survey says that only 48% "expect" to be done by December 31st, 1999. Apparently some of them who thought they were "done" at the beginning of the year now aren't even expecting to be done by the end!
But even worse is the little word "critical" that somehow slipped into the survey results between the earlier and later surveys. Far from being "done" at the end of last year, or even this year, it's only their critical systems that are even being talked about anymore. What about their "noncritical" systems? What percentage of their systems are "critical"? How do they decide which are critical? We don't know the answer to any of these questions, so their reported statistics are meaningless. However, it is clear that anyone who isn't going to be finished with their "critical" systems has given up on the non-critical systems. In other words, the situation must be much worse than it appeared to be when they were talking about completion dates for all of their systems.
-- Steve Heller (stheller@koyote.com), August 12, 1999.
As for your first point: Please note that although the earlier survey said that 55% were "done" as of January 1st, the later survey says that only 48% "expect" to be done by December 31st, 1999. Apparently some of them who thought they were "done" at the beginning of the year now aren't even expecting to be done by the end!Steve, please reread the first article:
Seventy-four percent of major corporations had expected to have more than half of their code "completely tested and compliant" by January 1 of this year, according to the last quarterly survey issued in December. But the current tracking poll reveals that only 55 percent actually reached this goal.
The first is talking of a percentage expecting to have "more than half" their code done; the second is talking of those expecting "all" their code done.
This a tracking poll, which began I believe in 1995. As such, it is extremely doubtful that they have "changed" the questions. Cap Gemini has included the term "critical" in the past, as well as omitting it.
In any case, you seem to be focussing far too much on this one survey. I picked it because it was the most pessimistic I found, and it also backed up the 66% I used. Capers Jones direct estimates also make this a conservative estimate. And, as I demonstrated in the previous post, applying the history of IT project completions also makes this 66% a conservative estimate.
-- Hoffmeister (hoff_meister@my-deja.com), August 13, 1999.
Steve, please reread the first article: Seventy-four percent of major corporations had expected to have more than half of their code "completely tested and compliant" by January 1 of this year, according to the last quarterly survey issued in December. But the current tracking poll reveals that only 55 percent actually reached this goal. The first is talking of a percentage expecting to have "more than half" their code done; the second is talking of those expecting "all" their code done.
You're correct; I misread that part of the article. However, that doesn't change the basic point, which is that their estimate of when they would be a certain percentage completed with their project was way off; 74 percent expected to be halfway done by a particular date, and only 55 percent claimed that they were halfway done by that date. This is especially telling when you consider that it's very difficult to determine when a software project is half done: many software projects are "almost done" for most of their entire lifespan.
This a tracking poll, which began I believe in 1995. As such, it is extremely doubtful that they have "changed" the questions. Cap Gemini has included the term "critical" in the past, as well as omitting it.
I don't think this is something that you should guess about. Do you know for a fact whether they have changed the question? If not, why can't they even report their own survey questions and answers correctly? If so, my point stands.
In any case, you seem to be focussing far too much on this one survey. I picked it because it was the most pessimistic I found, and it also backed up the 66% I used. Capers Jones direct estimates also make this a conservative estimate. And, as I demonstrated in the previous post, applying the history of IT project completions also makes this 66% a conservative estimate.
The reason I'm focusing on this survey is that it is your source. Clearly, if their reporting of their latest question and the latest results is correct, whether or not they previously had used the term "critical", there is no reason whatsoever for us to believe that these companies are going to finish their remediation in time. The definition of "critical" is subjective, which renders the statistics in question invalid. Therefore, the only prudent course of action is to prepare for the consequences of failed remediation by many large companies.
-- Steve Heller (stheller@koyote.com), August 13, 1999.
I don't think this is something that you should guess about. Do you know for a fact whether they have changed the question? If not, why can't they even report their own survey questions and answers correctly? If so, my point stands.Without having hardcopy of the actual survey, I cannot state unequivocally that the question has not changed.
However, you base your point on the fact that the term "critical" appeared in the latest release. Going back through previous releases by Cap Gemini:
Critical NOT used.
Critical NOT used.
Critical NOT used.
Critical IS used.
Critical NOT used.
Critical IS used.
Critical NOT used.
Critical IS used.
So, unless you believe they change the question every quarter or so, there is no doubt that the use of the term "critical" in the latest release does not indicate they have changed the question.
The reason I'm focusing on this survey is that it is your source. Clearly, if their reporting of their latest question and the latest results is correct, whether or not they previously had used the term "critical", there is no reason whatsoever for us to believe that these companies are going to finish their remediation in time. The definition of "critical" is subjective, which renders the statistics in question invalid. Therefore, the only prudent course of action is to prepare for the consequences of failed remediation by many large companies.
As I said previously, the analysis assumes in no way that any organization fully completes their remediation on time.
The analysis merely makes the assumption that, in total, 66% of the function points with Year 2000 errors are either replaced or remediated.
No single survey is the basis for that assumption. This survey provides some support. Previously referenced estimates by Capers Jones does as well. Applying the history of Software Project completions, which you suggested, makes this a conservative estimate. The referenced survey by IDC makes this a very conservative estimate as well.
The analysis is an attempt to determine just what the consequences of partial completion of remediation will be. The results show that the consequences, in all liklihood, will be no worse than what we are currently experiencing.
-- Hoffmeister (hoff_meister@my-deja.com), August 13, 1999.
The analysis merely makes the assumption that, in total, 66% of the function points with Year 2000 errors are either replaced or remediated.
I guess I haven't been clear about my main problem with your analysis. The question is what that 66% is a percentage of. If it is of "critical systems", however defined, as it appears to be, what percentage of total systems does that represent? Without knowing that, I don't understand how anyone can draw conclusions as to the percentage of function points that are or will be remediated. Do you know the answer to this question?
-- Steve Heller (stheller@koyote.com), August 13, 1999.
No, the analysis is not based only on "critical" systems. The analysis is of active applications with Y2k errors.The only reference to "critical" systems is from the addition of the term in the Cap Gemini survey. And that still seems open to conjecture. From this article Some Fortune 500 Y2K Late:
Fewer than half of US Fortune 500 companies expect all of their computer systems to be ready for Year 2000-related failures, in part because they are devoting much of their attention to ensuring that their top "mission-critical" systems are Y2K-compliant, a new survey has found.
So, even this survey seems open to interpretation.
The analysis is meant to estimate error rates. As such, it deals with active applications. Non-active or dormant applications do not have the potential to generate any appreciable errors for Y2k.
As stated, the 66% represents the percentage of active applications with Y2k errors, that are either replaced or put through remediation.
33% then represents the percentage of active applications with Y2k errors that go completely untouched. This would include active, non-critical systems, as well as critical systems not remediated.
Error rates are based on function points. An underlying assumption then is made that function points are evenly distributed between systems, which again is conservative. Though not an exact correlation, larger systems tend to be more critical, and thus are more likely to fall in the 66%.
As for what percentage of all systems are "critical", this without a doubt varies from site to site. Ed Yourdon in this article uses between 50% and 66% of systems being "critical".
But a large portion of those systems deemed "non-critical" are in fact "dormant", or non-active systems. Yourdon makes this point in the referenced article. In this Capers Jones paper, The Global Economic Impact of the Year 2000 Software Problem, a study of IBM data centers found between 40% and 70% of the applications could be classified as "dormant", or not being run in over a year. While IBM may not be completely representative, a large portion of systems labelled "non-critical" are in fact those found to be dormant.
-- Hoffmeister (hoff_meister@my-deja.com), August 13, 1999.
No, the analysis is not based only on "critical" systems. The analysis is of active applications with Y2k errors.
The only reference to "critical" systems is from the addition of the term in the Cap Gemini survey. And that still seems open to conjecture.
So now you are claiming that it is merely "conjecture" whether the percentage completion figures in the Cap Gemini studies refer to only "critical" systems or to all systems, even though they use the modifier "critical" in half of their reports? Clearly, if you can't even agree on the significance of the figures from the source that you yourself cite, there's no point in continuing the discussion any further.
-- Steve Heller (stheller@koyote.com), August 13, 1999.
Steve, we have spent far too much time on this one survey.You raised a valid question regarding the survey. Without having the actual survey and definitions, no definitive answer can be made.
To further the discussion, completely disregard the survey, as being inconclusive. I have supplied multiple other sources as backup for the figure this survey referenced, including the sources you yourself suggested.
Is this the only problem you have with my post?
-- Hoffmeister (hoff_meister@my-deja.com), August 13, 1999.
Continued....---------
Steve, we have spent far too much time on this one survey. You raised a valid question regarding the survey. Without having the actual survey and definitions, no definitive answer can be made. To further the discussion, completely disregard the survey, as being inconclusive.
Okay, fine with me.
I have supplied multiple other sources as backup for the figure this survey referenced, including the sources you yourself suggested.
Could you please give me those sources again? I've re-read the entire thread and haven't been able to find any reference to surveys containing any actual information about percentage of completed remediation, other than the Cap Gemini survey that we have already beaten into the ground. All I've seen is a couple of slides and articles that mention in passing the number of companies that do or don't "expect" to make the deadline, with no actual information about how far along they are now. Do you have any references to any surveys that actually ask the companies what they have completed, other than the Cap Gemini surveys?
Is this the only problem you have with my post?
The problem I have with your post is that your estimates of the number of companies that will finish any particular percentage of the remediation is so far unsupported by evidence, as far as I can tell. I'll be happy to examine the evidence, as soon as you provide it.
-- Steve Heller (stheller@koyote.com), August 13, 1999.Well, since you suggested it, let's look at using the history of IT projects.These metrics support an estimate of 80% or greater. 66% then leaves a fairly large margin of error.
-- Hoffmeister (hoff_meister@my-deja.com), August 13, 1999.Well, since you suggested it, let's look at using the history of IT projects. These metrics support an estimate of 80% or greater. 66% then leaves a fairly large margin of error.
That's very nice, but as we've already seen, the question is 80 percent of what? Have the companies actually been doing the work required to get their systems fixed? Is there any way we can find out this information? Apparently, there is no way to find this out, or at least neither you nor I can come up with one.
The problem is that trying to use statistics like "80 percent completion rates" assumes the very thing that we're trying to find out here (or that I'm trying to find out, at least): have they been working on all their systems, or just their critical systems, or just their most critical systems, or just their "top" mission critical systems, or what? You can get a different answer to that question from every survey.
Unlike "normal" projects, which can be laid out to a timetable selected by the company, Y2K projects have a fixed, immovable deadline. If companies didn't start all of their projects on time, and there seems to be no way we can tell whether they did, they almost certainly won't finish on time.
I guess the only conclusion we can draw from this discussion is that there is no way to find out any useful information about how much of the remediation companies are going to finish before the ball drops. Under such circumstances, the only prudent course is to take precautions against massive failures to remediate.
-- Steve Heller (stheller@koyote.com), August 13, 1999.First, your comment regarding "normal" projects flies in the face of my experience. Virtually every project I've been involved with, for good or bad, has been worked backwards from a given target date.Second, the distinction between "critical" and "non-critical" systems matters very little to the analysis. The vast bulk of "non-critical" systems are, in fact, dormant or inactive. Capers Jones uses a general estimate of 50% of a software portfolio as being inactive (source), which corresponds with the previous study mentioned by IBM.
Unless you have other information, every Y2k Project at the least addresses critical systems. Using the past metrics on software project completion yielded somewhere above 80% of the Projects being completed. Note this again is very conservative; a project not completely finished does not yield a 0% of applications addressed. Dropping this down to 66% of total active applications takes into account the non-critical applications.
Further support is found from Capers Jones, who uses an average of 75% in his Best Case scenario for overall errors addressed. Again providing support that 66% is at least in the ballpark.
Note this is not an exact estimate, which you seem to require. Again, I've provided support for the number I used. While questioning it, you have provided no evidence the number is any less. But use even 50%, if you wish. The result changes very little, and the conclusion changes not at all; that the current error rate being experienced right now is of the same magnitude as what can be expected at the rollover.
-- Hoffmeister (hoff_meister@my-deja.com), August 14, 1999.First, your comment regarding "normal" projects flies in the face of my experience. Virtually every project I've been involved with, for good or bad, has been worked backwards from a given target date.
While I've been involved in some projects like that, all of which were vastly late if not canceled entirely, the vast majority of the ones I've been involved with have actually been estimated in some reasonable manner. As this indicates, any project "estimated" like that has approximately 100 percent chance of being late, as the "estimate" has no relation whatsoever to the amount of time it actually takes to do the project. I believe any reputable source in the information-processing field would agree with me.
Second, the distinction between "critical" and "non-critical" systems matters very little to the analysis. The vast bulk of "non-critical" systems are, in fact, dormant or inactive. Capers Jones uses a general estimate of 50% of a software portfolio as being inactive (source), which corresponds with the previous study mentioned by IBM.
Sorry, your link points to the front page of the large site, not to any specific information about dormant systems. I'll be happy to look at your evidence, if you provide a valid link.
Unless you have other information, every Y2k Project at the least addresses critical systems. Using the past metrics on software project completion yielded somewhere above 80% of the Projects being completed. Note this again is very conservative; a project not completely finished does not yield a 0% of applications addressed. Dropping this down to 66% of total active applications takes into account the non-critical applications.
I have no doubt that they are addressing critical systems. However, I still haven't seen any evidence whatsoever that critical systems make up the vast majority or even any majority of all systems that need to be remediated. Let's take a look at what Ed Yourdon has to say about this issue as regards the federal government's systems:
But even the most passionate optimists find it difficult to argue that the non-mission-critical systems will be repaired in time; after all, the Federal government will barely be able to finish repairing its 6,399 mission-critical systems in time, and has had virtually nothing to say about the fate of another 66,000 non-critical systems.
While I have no specific information about the prevalence of noncritical systems in industry, I have no reason to believe that is vastly different from this proportion, which would mean that noncritical systems are over 90 percent of the total. If you have some evidence to the contrary, please present it.
Further support is found from Capers Jones, who uses an average of 75% in his Best Case scenario for overall errors addressed. Again providing support that 66% is at least in the ballpark.
Where does he get his information? Did he do a survey? Do you have a link to anything more than a slide or sentence in an article??
Note this is not an exact estimate, which you seem to require. Again, I've provided support for the number I used. While questioning it, you have provided no evidence the number is any less. But use even 50%, if you wish. The result changes very little, and the conclusion changes not at all; that the current error rate being experienced right now is of the same magnitude as what can be expected at the rollover.
Okay, I've provided some evidence that the proportion of critical systems is less than 10 percent. Does that change your conclusion any?
-- Steve Heller (stheller@koyote.com), August 14, 1999.Sorry, your link points to the front page of the large site, not to any specific information about dormant systems. I'll be happy to look at your evidence, if you provide a valid link.Sorry. The link should be here.
As well, this essay references the IBM study that found between 40% and 70% of a software portfolio inactive or dormant.
Where does he get his information? Did he do a survey? Do you have a link to anything more than a slide or sentence in an article??
Unfortunately, Steve, the internet does not provide full access to all information. But some more info:
This slide gives a fairly detailed breakdown, by industry, of the support for his 85% figure. Again, it would not appear he is simply pulling a figure out of the air.
Capers Jones here also makes the statement:
Given the strong probability that somewhere between 10% and perhaps 35% of potential year 2000 software problems will still be present at the dawn of the next century, it is now time to begin to start contingency planning for minimizing the damages which unrepaired year 2000 problems might cause.
So his high-end estimate matches my assumption of 66% addressed.
I have no doubt that they are addressing critical systems. However, I still haven't seen any evidence whatsoever that critical systems make up the vast majority or even any majority of all systems that need to be remediated. Let's take a look at what Ed Yourdon has to say about this issue as regards the federal government's systems:...
While I have no specific information about the prevalence of noncritical systems in industry, I have no reason to believe that is vastly different from this proportion, which would mean that noncritical systems are over 90 percent of the total. If you have some evidence to the contrary, please present it.
Okay, I've provided some evidence that the proportion of critical systems is less than 10 percent. Does that change your conclusion any?
Seriously? You're actually making the statement that you have no reason to believe the Federal Government is less efficient that business?
But OK, let's take a look even here.
What you have provided is evidence that 10% of the total Federal systems are being addressed as "critical". Not of active systems that require remediation or replacement, which is what the analysis is based upon.
The previous links support a range of 40-70% of applications being "dormant" or "inactive". My guess would be the Fed is at the high-range here, but let's just use Jones' general estimate of 50%.
The latest OMB report lists 6,399 systems identified as "mission-critical". Using your 90%, this approximates a total of 64,000 total systems. Applying Jones' 50% yields a total of 32,000 "active" systems.
The analysis starts with the Universe of applications non-compliant at the beginning of 1998. Some percentage of applications are already compliant. From the OMB report at the beginning of 1998, available here, 40% of the mission-critical systems are either compliant or being retired. Leaving 60% of systems requiring remediation or replacement.
Applying this percentage to the 32,000 "active" applications leaves 19,200 active, non-compliant systems.
So the percentage of active, non-compliant systems being addressed by the Federal Government as "mission-critical" would be approximately 33%.
Note also that the Fed is not only addressing "mission-critical" systems. A quick review of the latest OMB report here lists a number of agencies complete with "non-critical" systems as well.
So, even for the Federal Government, support for using at least a 50% number for active, non-compliant systems addressed can be found.
-- Hoffmeister (hoff_meister@my-deja.com), August 14, 1999.Okay, let's assume for the moment that your estimate of 66% of "systems addressed" is correct. Let's even assume that most of those systems will be remediated more or less successfully, even though this is a vastly different statement. You've made an estimate of the proportion of errors that will be experienced this year rather than next, and concluded that we are already seeing as many or more errors this year as we will early next year. Since the current error rate is being handled by IT, you conclude that next year will be more of the same.But you're still missing an important point : the types of errors that are experienced this year vs. next. Since it is not yet 2000, the types of errors that would have occurred so far are lookahead errors, not real-time information system errors. As has been discussed at length in c.s.y2k and other venues, the so-called "Jo Anne effect" errors, which are of this sort, are relatively easy to handle. This is because they do not impact the day-to-day functioning of the organization. Of course, it is important to be able to balance the books of the organization, but failure to do that or difficulty in doing that is not a show-stopper in most cases. That is why I, for one, have not predicted massive publicly visible IT problems this year.
Next year, we will see a new class of errors: inability to properly process live, real-time or near-real-time information with post-1999 dates. This is the sort of error that can bring an organization to a screeching halt. Even if the actual rate of errors is the same as this year, the seriousness of this type of error will be much greater than that of the lookahead errors. Therefore, I feel safe in predicting tremendous IT systems problems next year, always assuming that the lights are on and the other infrastructure pieces are working.
-- Steve Heller (stheller@koyote.com), August 14, 1999.Okay, let's assume for the moment that your estimate of 66% of "systems addressed" is correct. Let's even assume that most of those systems will be remediated more or less successfully, even though this is a vastly different statement. You've made an estimate of the proportion of errors that will be experienced this year rather than next, and concluded that we are already seeing as many or more errors this year as we will early next year. Since the current error rate is being handled by IT, you conclude that next year will be more of the same.OK. But realize the analysis did account for estimated rates of missing errors in remediated systems, and bad fixes.
But you're still missing an important point : the types of errors that are experienced this year vs. next. Since it is not yet 2000, the types of errors that would have occurred so far are lookahead errors, not real-time information system errors. As has been discussed at length in c.s.y2k and other venues, the so-called "Jo Anne effect" errors, which are of this sort, are relatively easy to handle. This is because they do not impact the day-to-day functioning of the organization. Of course, it is important to be able to balance the books of the organization, but failure to do that or difficulty in doing that is not a show-stopper in most cases. That is why I, for one, have not predicted massive publicly visible IT problems this year.
No, Steve. A very small percentage of errors expected this year were of the "look-ahead" or "JAE" type, less than 10%. I agree, these errors have been vastly overblown.
The bulk of the errors I'm considering are the errors due to system implementations, and account for virtually every instance of system problems associated as "Y2k-Related" to date.
Next year, we will see a new class of errors: inability to properly process live, real-time or near-real-time information with post-1999 dates. This is the sort of error that can bring an organization to a screeching halt. Even if the actual rate of errors is the same as this year, the seriousness of this type of error will be much greater than that of the lookahead errors. Therefore, I feel safe in predicting tremendous IT systems problems next year, always assuming that the lights are on and the other infrastructure pieces are working.
While the seriousness of rollover errors may be much greater than look-ahead problems, the seriousness of implementation errors at the very least are on par with possible rollover errors, and in fact probably exceed rollover errors in level of seriousness.
Even so, I have heavily discounted implementation errors, by 85%, to provide an even greater margin of error. The analysis merely assumes that only 15% of the errors due to implementations are on par with all Y2k rollover errors, an assumption I think of as extremely conservative.
-- Hoffmeister (hoff_meister@my-deja.com), August 14, 1999.While the seriousness of rollover errors may be much greater than look-ahead problems, the seriousness of implementation errors at the very least are on par with possible rollover errors, and in fact probably exceed rollover errors in level of seriousness.
Yes, perhaps, but the implementation errors that you are referring to are different from rollover errors in one important way: rollover errors will not occur until next year. Therefore, however many of them exist, they will all show up next year. This means that attempts to determine what percentage of problems have already been seen or will be seen before the end of the year cannot succeed unless and until we know how many rollover errors there are. This is impossible to determine, and therefore any attempt to calculate the percentage of problems have already been seen is doomed to failure.
-- Steve Heller (stheller@koyote.com), August 14, 1999.Now I want to revisit another point that I made to which you have not responded:
First, your comment regarding "normal" projects flies in the face of my experience. Virtually every project I've been involved with, for good or bad, has been worked backwards from a given target date.
While I've been involved in some projects like that, all of which were vastly late if not canceled entirely, the vast majority of the ones I've been involved with have actually been estimated in some reasonable manner. As this indicates, any project "estimated" like that has approximately 100 percent chance of being late, as the "estimate" has no relation whatsoever to the amount of time it actually takes to do the project. I believe any reputable source in the information-processing field would agree with me.
Let me add a little bit more to my reply to your comments: are you saying that no matter what date is arbitrarily assigned as the ending date of a project, the likelihood of the project's successful completion is the same? For example, let's suppose that we have the project to rewrite all of the IRS's information systems, and the deadline is nine months from now. Would such a project have the same chance of completion as if the ending date were nine years later? Or, for that matter, if the ending date doesn't matter, why not make it tomorrow?
Of course, the point I'm making is that, without any information about the means of estimating how long a project is supposed to take, there is no way to know whether or not it is likely to be done on time, or how late it is likely to be. I've tried to find references on the Internet for studies on the effects of arbitrarily assigned ending dates on the likelihood of successful project completion within those dates. So far, I haven't had any luck. However, commonsense and experience indicate that projects with arbitrarily assigned ending dates will be finished on time only on the rarest occasions, if ever. The very notion of estimation of how long a project will take implies that the ending date cannot be determined arbitrarily, but must be calculated according to estimates of how long each individual section of the project will take. Any other means of "estimation" is merely political in nature, and has nothing to do with reality.
The naive answer to this, of course, is that if you have to get the project done in a fixed amount of time, you just add manpower to make the "man-months" calculation come out properly. However, equally of course, this is a well-known fallacy. I'm sure you're aware of the book The Mythical Man-Month, by Frederick Brooks, which explodes this fallacy once and for all. One of his most relevant comments is (from memory, so it may not be word for word): More software projects have gone astray from lack of calendar time than for any other cause, gross incompetence included.
-- Steve Heller (stheller@koyote.com), August 14, 1999.Yes, perhaps, but the implementation errors that you are referring to are different from rollover errors in one important way: rollover errors will not occur until next year. Therefore, however many of them exist, they will all show up next year. This means that attempts to determine what percentage of problems have already been seen or will be seen before the end of the year cannot succeed unless and until we know how many rollover errors there are. This is impossible to determine, and therefore any attempt to calculate the percentage of problems have already been seen is doomed to failure.Estimating the potential error rate due to Y2k is actually fairly straightforward. Capers Jones references 15% of function points containing Y2k errors. Other studies, for example by Howard Rubin, cite findings of 30 date references per 1000 lines of code, or 3%. The potential error rate for Y2k errors in code is far from an unknown phenomenon.
As well, the fact that system implementations do cause high rates of errors is undeniable.
While it is no doubt impossible to determine exact figures on error rates, it is not impossible to demonstrate that the potential error rates are in line with what we are currently experiencing.
As for target dates. The idea that target dates are based on detailed assessments of each individual section, while idealistic, has very little to do with reality.
At best, a large scale project begins with a small number of people doing very broadbrush scoping. Depending on size, this lasts between 2 weeks and a month. The result is a budget estimate, and a target date. All long before indvidual tasks are determined and scoped. And this target date remains, except in extreme circumstances.
I did not say this was ideal, or good. Just reality.
The actual tasks are then scoped backwards from the date. Yes, I'm familiar with the "Mythical Man Month". I'm also very familiar with the reality software implementations in large corporations.
In using the software metrics for project completions, I was again very conservative. These should be scaled to reflect the fact we are speaking of remediation projects, and not development projects. But I left them untouched, again to provide a larger margin for error.
-- Hoffmeister (hoff_meister@my-deja.com), August 14, 1999.Estimating the potential error rate due to Y2k is actually fairly straightforward. Capers Jones references 15% of function points containing Y2k errors. Other studies, for example by Howard Rubin, cite findings of 30 date references per 1000 lines of code, or 3%. The potential error rate for Y2k errors in code is far from an unknown phenomenon.
Can you provide some research results to indicates the proportion and seriousness of century rollover errors? When have we had a previous century rollover on which to do the research?
I notice that you did not answer my point that no rollover errors have occurred yet, and therefore cannot be estimated by looking at the number so far encountered. Do you have a response to that?
At best, a large scale project begins with a small number of people doing very broadbrush scoping. Depending on size, this lasts between 2 weeks and a month. The result is a budget estimate, and a target date. All long before indvidual tasks are determined and scoped. And this target date remains, except in extreme circumstances.
While not absolutely ideal, this is still a far cry from deciding the target date completely arbitrarily in advance of any analysis at all, as has been done with every Y2K project that started too late (almost certainly a large proportion of the whole, although again it is impossible to quantify the proportion). Of course, other projects have been "estimated" in the same way, and as I have pointed out, in such a case, there is NO reason to believe that the target date has ANY relationship to how long the project will actually take. Virtually all such projects that I have been involved with have run far over their target dates, and I see no reason to believe that Y2K projects are mysteriously immune to this problem.
-- Steve Heller (stheller@koyote.com), August 14, 1999.
-- Hoffmeister (hoff_meister@my-deja.com), August 17, 1999.
Last one...-----------
Can you provide some research results to indicates the proportion and seriousness of century rollover errors? When have we had a previous century rollover on which to do the research?As for the proportion of Y2k errors, again, Capers Jones' work estimates at 15% of function points. Howard Rubin cites an extensive study in his article, More Millenium Metrics, where 3% of the code base contains date references.
As for seriousness, these run the gamut from trivial to abends. We aren't talking of reading tea-leaves, here. Running programs through a "time-machine" system provides fairly precise answers as to how an application responds to 2000 dates. It is not necessary to have previously experienced a century rollover, to know how an application reacts. Or more precisely, applications have experienced the century rollover on thousands of test machines.
I notice that you did not answer my point that no rollover errors have occurred yet, and therefore cannot be estimated by looking at the number so far encountered. Do you have a response to that?
I didn't respond, because the estimate of rollover errors is based in no way on the number of errors encountered so far. It is based on fairly well-documented studies of the potential for rollover errors.
And again, target dates. The analysis does not depend on any entity's project completing on time. This is not an all or nothing situation. A company may miss the deadline for completing all of its systems, but still complete a substantial portion of applications.
-- Hoffmeister (hoff_meister@my-deja.com), August 14, 1999.Running programs through a "time-machine" system provides fairly precise answers as to how an application responds to 2000 dates. It is not necessary to have previously experienced a century rollover, to know how an application reacts. Or more precisely, applications have experienced the century rollover on thousands of test machines. Okay, that is a valid point. I will accept time machine simulation as a proxy for real operation after rollover, of course with the usual caveat that testing and production are not identical.
And again, target dates. The analysis does not depend on any entity's project completing on time. This is not an all or nothing situation. A company may miss the deadline for completing all of its systems, but still complete a substantial portion of applications.
My point does not rely on assuming that a company must fix all of its systems. The question is whether setting a projected completion date that is fixed by the calendar rather than by any analysis of the problem reduces the likelihood of achieving any particular state of completion. Both commonsense and experience indicate very strongly that such completion dates are much more likely to be missed by a large margin than completion dates set after even a cursory analysis of the problem (even though the latter are often overrun as well). Do you disagree with this point? If so, on what basis?
Okay, I accept that as a valid way of estimating the seriousness of rollover errors, of course with the caveat that testing and production are not the same.
-- Steve Heller (stheller@koyote.com), August 15, 1999.Running programs through a "time-machine" system provides fairly precise answers as to how an application responds to 2000 dates. It is not necessary to have previously experienced a century rollover, to know how an application reacts. Or more precisely, applications have experienced the century rollover on thousands of test machines.
Okay, that is a valid point. I will accept time machine simulation as a proxy for real operation after rollover, of course with the usual caveat that testing and production are not identical.
And again, target dates. The analysis does not depend on any entity's project completing on time. This is not an all or nothing situation. A company may miss the deadline for completing all of its systems, but still complete a substantial portion of applications.
My point does not rely on assuming that a company must fix all of its systems. The question is whether setting a projected completion date that is fixed by the calendar rather than by any analysis of the problem reduces the likelihood of achieving any particular state of completion. Both commonsense and experience indicate very strongly that such completion dates are much more likely to be missed by a large margin than completion dates set after even a cursory analysis of the problem (even though the latter are often overrun as well). Do you disagree with this point? If so, on what basis?
-- Steve Heller (stheller@koyote.com), August 15, 1999.My point does not rely on assuming that a company must fix all of its systems. The question is whether setting a projected completion date that is fixed by the calendar rather than by any analysis of the problem reduces the likelihood of achieving any particular state of completion. Both commonsense and experience indicate very strongly that such completion dates are much more likely to be missed by a large margin than completion dates set after even a cursory analysis of the problem (even though the latter are often overrun as well). Do you disagree with this point? If so, on what basis?If the calendar date is arbitrary, I'd agree.
The year 2000 is not an arbitrary, unknown date. It allowed organizations to work backwards from a date, to determine when actual remediation had to begin. It also allowed a buffer to be built in, such as the 12/31/98 date you mentioned previously, and a variety of 1999 dates, with probably June-July being most prevalent, to allow for schedule overruns.
To varying levels of degree, we have seen evidence that projects with similiar date constraints, at the very least, follow past experience, if not eclipsing it. The Euro implementation, while not without problems, demonstrated this fact. The airline reservations systems as well. Even the state benefit systems that Yourdon mentions at least mirror past experience with software projects.
The fact that the predicted shortage in COBOL programmers failed to materialize points this out, as well as the declining business of "remediation specialists". While I won't question that some organizations may have started too late, and are truly in a struggle, the evidence does not point to this being the case in general.
-- Hoffmeister (hoff_meister@my-deja.com), August 16, 1999.If the calendar date is arbitrary, I'd agree.
The year 2000 is not an arbitrary, unknown date. It allowed organizations to work backwards from a date, to determine when actual remediation had to begin. It also allowed a buffer to be built in, such as the 12/31/98 date you mentioned previously, and a variety of 1999 dates, with probably June-July being most prevalent, to allow for schedule overruns.
To varying levels of degree, we have seen evidence that projects with similiar date constraints, at the very least, follow past experience, if not eclipsing it. The Euro implementation, while not without problems, demonstrated this fact. The airline reservations systems as well. Even the state benefit systems that Yourdon mentions at least mirror past experience with software projects.
The fact that the predicted shortage in COBOL programmers failed to materialize points this out, as well as the declining business of "remediation specialists". While I won't question that some organizations may have started too late, and are truly in a struggle, the evidence does not point to this being the case in general.
I'd like to see any evidence that you have that most organizations took Y2K seriously enough to start their projects sufficiently before the deadline that they had any reasonable chance of making it, where "making it" means that they finished enough of their projects to survive as a corporation. I haven't seen any evidence like that; in fact, as far as I can tell, most organizations did not start their Y2K projects in earnest until 1997 or even 1998. Given the decades of previously accumulated software that they had to fix or replace, I have great difficulty believing that a year or two would do the job, especially when they have devoted only a fraction of their IT resources to the project.
As for the meaning of the apparently ample supply of Cobol programmers, I'd draw a diametrically opposite conclusion from it: that the organizations have given up. Otherwise, we should have seen a significant number of large organizations announcing completion of their Y2K projects. So far, I haven't seen more than perhaps one or two such announcements. For the rest, I have seen statements that they are "working on it, and on track". If you know where I could find any sizable number of statements by large organizations that they are done with their Y2K projects, I would greatly appreciate your providing the URL(s). Otherwise, I'll have to conclude that almost no large organizations are finished yet.
-- Steve Heller (stheller@koyote.com), August 16, 1999.I'd like to see any evidence that you have that most organizations took Y2K seriously enough to start their projects sufficiently before the deadline that they had any reasonable chance of making it, where "making it" means that they finished enough of their projects to survive as a corporation. I haven't seen any evidence like that; in fact, as far as I can tell, most organizations did not start their Y2K projects in earnest until 1997 or even 1998. Given the decades of previously accumulated software that they had to fix or replace, I have great difficulty believing that a year or two would do the job, especially when they have devoted only a fraction of their IT resources to the project.I would agree that remediation efforts probably did not start in earnest until 1998. But companies began replacing systems before that. For example, look at SAP. In 1994, there were about 1,000 installations. Today, more than 20,000. SAP's revenues more than tripled from 1995 to 1998. And the SAP market has been fuelled, in a very large part, by Y2k. Even a cursory review of corporate Y2k disclosures finds SAP mentioned throughout.
As for the meaning of the apparently ample supply of Cobol programmers, I'd draw a diametrically opposite conclusion from it: that the organizations have given up.
Yes, I've heard this statement before, and find absolutely no credibility in it.
I can literally point to thousands of pieces of information that say corporations are working on the problem.
Can you cite even one reference to a company that has "given up"? Do you truly believe that a corporation would, even at this late stage, just throw their hands in the air and stop working on the problem? I certainly haven't seen it, and it absolutely does not fit with my experience with corporations in general.
Otherwise, we should have seen a significant number of large organizations announcing completion of their Y2K projects. So far, I haven't seen more than perhaps one or two such announcements. For the rest, I have seen statements that they are "working on it, and on track". If you know where I could find any sizable number of statements by large organizations that they are done with their Y2K projects, I would greatly appreciate your providing the URL(s). Otherwise, I'll have to conclude that almost no large organizations are finished yet.
No, and I doubt any organization will truly "finish" until well after the rollover. I expect testing and contingency planning to run almost to the rollover.
Year 2000 projects involve much more than just the remediation. Even applications completed must be continually change-controlled for possible Y2k problems. This is one of the reasons that active systems being remediated were held to last; it just isn't feasible for a large corporation to somehow "freeze" application modifications for a year or more in their active systems.
-- Hoffmeister (hoff_meister@my-deja.com), August 16, 1999.I would agree that remediation efforts probably did not start in earnest until 1998. But companies began replacing systems before that. For example, look at SAP. In 1994, there were about 1,000 installations. Today, more than 20,000. SAP's revenues more than tripled from 1995 to 1998. And the SAP market has been fuelled, in a very large part, by Y2k. Even a cursory review of corporate Y2k disclosures finds SAP mentioned throughout.
Yes, I'm sure that SAP has done very well with Y2K replacements. But I've still been unable to find any indication that very many companies had even started planning their Y2K effort before late 1997. Therefore, whatever systems were replaced before that with SAP implementations may have been okay, but the other systems weren't even being analyzed before that time. As far as I'm aware, SAP can only do a portion, possibly a minor fraction, of the systems in a very large corporation. Therefore, my comments about the serious hazards of making a late start still apply to those other systems.
Can you cite even one reference to a company that has "given up"? Do you truly believe that a corporation would, even at this late stage, just throw their hands in the air and stop working on the problem? I certainly haven't seen it, and it absolutely does not fit with my experience with corporations in general.
I should have been more clear. I'm not saying that anyone is going to stop working on it entirely. However, most companies appear to be expending a significant percentage of their information systems resources on other projects before their Y2K project is as finished as it can possibly be this year. Maybe this isn't giving up, but it certainly isn't taking the problem seriously enough.
No, and I doubt any organization will truly "finish" until well after the rollover. I expect testing and contingency planning to run almost to the rollover.
Year 2000 projects involve much more than just the remediation. Even applications completed must be continually change-controlled for possible Y2k problems. This is one of the reasons that active systems being remediated were held to last; it just isn't feasible for a large corporation to somehow "freeze" application modifications for a year or more in their active systems.
No, I can't agree with that. Although of course it would be painful, it would be entirely possible for them to freeze their application systems for a year, if the threat were taken seriously enough. Of course, that would require that they finish their Y2K modifications last year, which as far as I can tell almost no one has done. Apparently, top management is either unwilling or unable to understand the threat to the organization posed by Y2K, or unwilling or unable to convey this to the line managers. I have worked for number of large corporations, and unfortunately I find it very easy to believe that top management is out of touch with the people who could explain this to them.
-- Steve Heller (stheller@koyote.com), August 16, 1999.Yes, I'm sure that SAP has done very well with Y2K replacements. But I've still been unable to find any indication that very many companies had even started planning their Y2K effort before late 1997. Therefore, whatever systems were replaced before that with SAP implementations may have been okay, but the other systems weren't even being analyzed before that time. As far as I'm aware, SAP can only do a portion, possibly a minor fraction, of the systems in a very large corporation. Therefore, my comments about the serious hazards of making a late start still apply to those other systems.Other than very specialized, custom applications, SAP can handle the bulk of a large corporations applications. This is the very reason SAP has become such a ubiquitous solution, especially as it relates to Y2k.
I should have been more clear. I'm not saying that anyone is going to stop working on it entirely. However, most companies appear to be expending a significant percentage of their information systems resources on other projects before their Y2K project is as finished as it can possibly be this year. Maybe this isn't giving up, but it certainly isn't taking the problem seriously enough.
Again, Y2k is not an unknown problem. Corporations are moving on to other projects because they feel they have their Y2k work well in hand.
No, I can't agree with that. Although of course it would be painful, it would be entirely possible for them to freeze their application systems for a year, if the threat were taken seriously enough. Of course, that would require that they finish their Y2K modifications last year, which as far as I can tell almost no one has done. Apparently, top management is either unwilling or unable to understand the threat to the organization posed by Y2K, or unwilling or unable to convey this to the line managers. I have worked for number of large corporations, and unfortunately I find it very easy to believe that top management is out of touch with the people who could explain this to them.
Anything is possible. The point is would it make sense?
In any case, I think we're beginning to stray. The original analysis did not make overly optimistic asumptions as to remediation. Originally I used 66%; even 50% doesn't change the conclusion. Yourdon uses I believe 80%; Jones acknowledges figures in the same range. I've provided backup for these numbers. And even companies whod did start late, will get some percentage of their applications completed.
-- Hoffmeister (hoff_meister@my-deja.com), August 16, 1999.Okay, I think we've covered this ground pretty thoroughly. I suggest that each of us write a summary of where we think we've ended up, post them, and then throw this into the public arena. What do you say?
-- Steve Heller (stheller@koyote.com), August 16, 1999.Fine by me. Go ahead when you're ready.
-- Hoffmeister (hoff_meister@my-deja.com), August 16, 1999.Here is my summary of the arguments and evidence presented on this thread. First, statements for which there is some supporting evidence that a reasonable person would consider indicative if not conclusive:
- There is some information about the likely effect of rollover errors, derived from time machine experiments. These experiments make it possible to determine the likely severity of rollover errors.
- There is a study cited by Hoffmeister that provides some information indicating that a fairly large percentage of systems that are not "critical" are "dormant" and therefore do not have to be remediated, based on data from studying IBM's software portfolio. However, that same study also stated that "it turned out to be very difficult to separate the active portions from the dormant portions of software portfolios", which reduces the positive effect of this factor.
- A number of large organizations have been replacing a significant portion of their software portfolio with packages like SAP, which would reduce the number of systems that would need to be remediated.
Second, here are the undefined and inconclusive elements of the situation:
- The original study by Cap Gemini on the progress of remediation in large companies cited by Hoffmeister has turned out to be ambiguous, in that the wording of the results does not clearly distinguish between "all systems" and "critical systems". As a result, he has abandoned it as a source. This leaves open the question of what stage in remediation the average company has reached or will reach by the end of the year.
- There is no study cited that provides reliable information about what percentage of systems are "critical systems". The only study cited was that of the federal government, in which they were less than 10 percent of all systems.
- There is no way of determining how companies decide which systems are "critical" and which systems are not. In fact, the distinction may be largely meaningless, as pointed out in an article cited by Hoffmeister in this discussion: "After all, a system that's non-critical to your organization may be very mission-critical to some of your external suppliers, vendors, or customers.
- There is no study cited that indicates the effect of an arbitrary ending date on the likelihood of success of any stage of remediation. However, experience and common sense indicates that such projects have a much lower chance of finishing even close to the target date than projects that have been analyzed in advance of setting the target date.
- There is no study cited which indicates that the average large company began its Y2K remediation task early enough to have any reasonable probability of finishing or even mostly finishing the task. However, we have agreed that most companies did not start the remediation in earnest until late 1997 or early 1998. This leaves them only a very short period of time to do a massive amount of remediation work, and cannot be construed positively.
What are we to make of all of this? I think the only reasonable conclusion to be drawn from this discussion is that there is no way for us to even reasonably accurately estimate the degree of remediation that large organizations will complete by the end of this year. Therefore, I conclude that it is only prudent to prepare for the possible consequences of a massive failure of remediation.
-- Steve Heller (stheller@koyote.com), August 17, 1999.The original analysis stands basically unchallenged.Various aspects have been questioned here, only to be immediately dropped.
One assumption has been questioned extensively, that being the percentage of remediation completed. The basis for this questioning has involved almost solely the difference between "critical" and "non-critical" systems. Almost by definition, this distinction is trivial to the analysis; "non-critical" systems are those the organization feels they can do business without.
Even so, no evidence has been cited that the percentage of remediation completed on active systems is any lower than the figure I used. The one attempt was to use the Federal Government, an admittedly poor example as compared to virtually any coproration. But an examination of even the Fed revealed that support for the figures I used can be found.
Other sources suggested also provide support. Using the history of past software projects results in a much higher percentage. Capers Jones uses a higher percentage in his analysis.
These are sources that have been used to support the more pessimistic analysis of the effects of Y2k.
The analysis is based in no way on optimistic assumptions. In fact, very pessimistic estimates have been used. Indeed, assuming that 50% of the active systems with Y2k errors remain completely untouched qualifies for extremely "massive failure in remediation". And the reason is simple; my opinion on Y2k is not based on an assumption that no problems will occur, but that those problems will be dealt with, as problems are being dealt with today.
The conclusion remains basically unchallenged: that we are currently experiencing IT systems error rates, simultaneously, that at the least are of the same magnitude as those which can be expected at the rollover to the Year 2000. These errors account for virtually every instance of "Y2k-related" errors to date, be it the World Bank, or MCI, or SUN Hydraulics. Without a doubt, implementation errors far exceed the severity of Year 2000 errors; even so, I discounted these errors by 85% within the analysis.
While these errors have caused problems, as system errors have in the past, they have not even begun to approach a level that would cause any form of overall collapse. Neither will the rollover to the Year 2000.
-- Hoffmeister (hoff_meister@my-deja.com), August 17, 1999.
Return to my main page