How do district court judges with varying levels of experience perform on claim construction? Are judges more likely to have their decisions affirmed when they have previous claim construction experience? Previously here and here I provided some background on the large database I compiled and some of the results. A draft of the paper, Practice Makes Perfect? An Empirical Study of Claim Construction Reversal Rates in Patent Cases, can be downloaded here.
Today’s post provides an analysis of the data based upon two measures of district court judge experience: (1) the size of the district court judge’s total patent docket over an eleven year period; and (2) the performance after a first reversal by the Federal Circuit. As for the first measure, I determined the reversal rate (as measured by cases in which an incorrect claim construction required the case to be reversed or vacated) based upon the number of patent lawsuits a district court judge was assigned. Figure X below illustrates the results.
Figure X shows that the claim construction reversal rate varies little with the total number of patent lawsuits handled. The reversal rate was between 26.7% and 34.1%.
I turn now to the second measure of experience. It may be that a district court judge pays little attention to a Federal Circuit opinion if the case was affirmed. In contrast, when the result of a case is affected, the district court judge takes note. A remanded case is often returned to the same district court judge for further proceedings. As these further proceedings are time-consuming, the district court judge likely will remember the opinion remanding a patent lawsuit to his or her docket. Figure N below illustrates the reversal rate on all subsequent claim construction appeals after a particular district court judge has been reversed or vacated at least once due to an erroneous claim construction.
Figure N shows that the first reversal does not have a significant effect on the future performance of the district court judges, with the rate the varying from 25.0% and 44.4% for reversals, and from 31.4% and 44.4% for errors. The percentages in this breakdown vary to a greater degree than the percentages based upon number of prior appeals (reported
yesterday). However, the trend is not linear and the differences among the judges illustrated in Figure N are not statistically significant.
< ?xml:namespace prefix ="" o />
The data illustrated above and described yesterday do not suggest that district court judges learn from having their claim construction rulings appealed. As I explain in more detail in the paper, there are many possible explanations for the surprising results. Three likely explanations are (i) that claim construction is inherently indeterminate; (ii) that district court judges are incapable of or not interested in learning how to perform claim construction; and (iii) that the Federal Circuit decisions do a poor job of teaching district court judges how to construe claims. (Another possible explanation is that limitations in the data, as described in detail in the paper, affect the results.) If the indeterminacy explanation is correct, then it will be difficult to lower the claim construction reversal rate, especially as long as those decisions are reviewed de novo. If the district court judge explanation is correct, then a primary rationale for adopting the Issa patent pilot program is likely faulty. In other words, if district court judges cannot learn how to perform claim construction, creating quasi-specialized patent judges is unlikely to reduce the reversal rate. If the Federal Circuit explanation is correct, then the Federal Circuit must redouble its efforts to create a uniform body of claim construction case law.
For those who are interested, the paper also analyzes the data based upon various other characteristics of the district court judges including age and years of judicial experience.
I’d like to thank Dennis for permitting me to share my results with the readers of Patently-O. I’d like to also thank the readers of Patently-O for their comments.
Nice try Mooney. You truly are a legend in your own mind. The only thing you are unwrapping from your finger is the bandage from that nasty crack pipe burn…
“you seem like you’re trying to bring about some kind of nonspecific yet dramatic change ”
On the contrary. I’m just watching the action and tossing in the occasional glib comment. You seem a tad touchy, CaveMan. Four comments in a row, in one thread, in response to a one and a half sentence comment from me? Geez.
[unwraps CaveMan from finger]
Go free, my son. There are so many new things in the world for you to discover, e.g., bronze.
Mooney, you seem like you’re trying to bring about some kind of nonspecific yet dramatic change based on making wild exaggerated characterizations of “shoveling” and “abuses” and “maelstroms” and what not, in a public forum where you think people who matter will notice. Sounds kind of like Frenkel.
Well documented abuses? How do you abuse a CAFC rule? Either there is a TSM or not. And what’s all this about a maelstrom? Are you on crack? Oh wait, you are.
Excuse me, TSM required a basis to support a combination.
Sorry Mooney, once again, the crack. Not sure what your points have been in the last posts. My position is quite defensible. KSR simply made required some basis to support a combination. Now what are the “false positives?” If anything, it suffered from false negatives. I guess that means that YOU are wrong – as usual.
Cavey writes “I disagree that it had false positives.”
That’s nice. Sadly, it’s also an indefensible position in light of the well-documented abuses of the strictly applied TSM shoveled by applicants and patentees and swallowed by district court judges, all leading up to the current patent maelstrom/meltdown.
JAOI, you have to understand the history of TSM, I think I wrote a lengthy missive on this before. TSM was a way to keep slovenly types from throwing claim terms into a search engine and cobbing together the first two or three references that popped up. It forced the Examiners and defendant’s counsel to actually read the references before attempting to apply them. I disagree that it had false positives. What it did was make it challenging to combine references. However, like I said, combinations can still be fairly easily challenged notwithstanding KSR. Provided, of course, that you’re not hanging around some lake smoking crack – Mooney.
Sorry Mooney, I can’t understand you, once again, through the crack cloud. I think I can get some backing on this, but you, my friend, are the only one on at Patently-O who is consistently preoccupied about being right all the time (even when it doesn’t really matter) A true tell tale of a hobbled personality. Someone like you can never learn and evolve.
anonymous writes “CaveMan, TSM is indeed a fairly established rule, but it had a lot of false positives. Too many bad patents were upheld.”
CaveMan knows this. He just doesn’t like being wrong.
CaveMan, TSM is indeed a fairly established rule, but it had a lot of false positives. Too many bad patents were upheld. So the Supreme Court replaced it with a flexible standard.
JAOI, KSR and eBay are indeed openings for result-oriented judging, but the same is true every time a rule is replaced with a standard.
jud, if it were possible, I think I’d take your bet. The best evidence of internal inconsistency at the CAFC is probably split panel decisions on claim construction issues. I’m not sure how many of those there have been since Phillips. There have been a few significant ones, but I’m nearly positive it’s not anything close to 30%
Hi jud,
A sucker’s bet for sure. Of course, you are right.
Do you think the comment at 10:30 above would begin to straighten things out some?
I’d wager 5 bucks that if you set up three panels of CAFC judges to review each other’s claim constructions in a blinded manner, they’d have reversal rates of 30-35%, too.
The problem isn’t the judges, it’s the system.
examiner#6k,
With all due respect, trust everyone, all judges, all presidents and their appointed officials, all examiners, all congress people, ford and the tobacco companies, and most of all, your doctors when they tell you to take you meds.
I repeat a comment made by me and others:
There is something wrong with the analysis. Fig. X shows that the reversal rate for judges handling 51-100 patent cases is 34.1%, and the overall reversal rate is over 25%. However, one out of every four cases disposed of by a court (either as a result of a jury verdict or decision of a judge) is NOT reversed. Less than half of disposed cases are appealed. I believe than no conclusion can be reached from the data discussed so far.
“it is obvious because I say it is. Like porn, I know it when I see it” Couldn’t have said it much better myself.
“With all due respect, eBay and KSR are indefensible – why would you want to defend the two horrendous decisions unless your bread is buttered by doing so?”
Why would you say they’re horrible and indefensible, unless your bread is buttered by doing so? This being said however, with very little, to no respect as you fail to respect the highest courts of this land in whom you put your trust in every single day of your life. If you can’t trust them, I’m going to go out on a very tiny limb here and say you can’t hardly trust anyone.
JAOI I hope that’s a Texas two-step…
Dear anonymous,
Re: “Cybor should be scrapped, whether by Congress, the Supreme Court, or the en banc CAFC.”
Okay, how about this: Give deference where deference is due. Which is what de novo review already does do. I know, I know, a lot of patent attorneys what to kill Cybor.
But why?, when the CAFC can and does give deference whenever they want to, and they don’t give deference when they don’t want to. Some district judges will get it wrong no matter what the standard of review is. The CAFC does whatever it wants to do anyway.
All judges can do that and sometimes do do that and most of the time get away with such do do. It is the do do that’s problematic, to say the least. If a judge(s) doesn’t like the direction the merits of a particular patent case points, they feel free to fudge, i.e., do do, if you catch my drift.
I don’t like to criticize without suggesting some improvement. How about this little two-step for starters:
1) The Supreme Court finds a case to reign in the CAFC and mandate that every CAFC decision be consistent with en banc Phillips and other generally accepted standards.
2) And when the CAFC wants to change its own precedent or any other precedent, make ’em do it en banc, not at the whim of one panel or another.
Excuse me anonymous, but KSR deconstructed a fairly well established rule. Therefore, your argument doesn’t hold water. KSR has given litigants new hope in cobbing together art that previously would have been difficult to combine in light of a more strict TSM test. However, savvy practitioners, examiners and attorneys alike knew how to negotiation the TSM straights. Even now, savvy attorneys can defeat a combination using KSR directly. There is no shortcut to understanding what you’re doing and doing it well. Sorry Mooney.
To me the interesting thing about the results are the relatively high percentage of reversals based on a median number of cases handled suggesting that the light dockets get it right and the heavy dockets get it right, but the split dockets seem to have a problem.
Dear anonymous,
With all due respect, if you are willing to be objective, your only regret will be the slap you give yourself on the forehead when you say: Of course, why didn’t I realize that sooner.
Re: “…it would be silly to say that statutory interpretation is ‘inherently indeterminate.’”
If you want to believe that, go ahead, but many fundamental reasons explaining why this is not just so have been presented repeatedly. I suggest you read the 1895 quote above as a start.
I mostly agree with you about Phillips. From what I see, Phillips is the best law to date on many claim construction issues. The vagaries of claim construction remain, however, because CAFC panels apply Phillips whimsically or ignore Phillips altogether when Phillips does not lead to the result they want. Possible reasons for this abound. The CAFC may be trying to appease the Supreme Court, the Congress, those in the Executive, big businesses, the media and the public, all of whom have been educated by lobbyists’ et al. to believe that self-employed inventors are scum.
and
Re: “I should say a couple of things in defense of (1) eBay and (2) KSR.”
(1) Haven’t you heard? The eBay decision is inconsistent with the Constitution. Last I herad, the Constitution trumps all.
(2) KSR made patent validity, in essence, “flexible”; the Supreme Court’s KSR decision is an open invitation for judges to render result-oriented patent decisions at whim — their own personal whim or big business influence whim or media stirred up whim-of-the-day.
Prior to KSR, the teaching, suggestion, motivation standard, TSM, required some form of tangible evidence to determine obviousness from non-obviousness. With KSR, the standard is closer to, “it is obvious because I say it is. Like porn, I know it when I see it.”
With all due respect, eBay and KSR are indefensible – why would you want to defend the two horrendous decisions unless your bread is buttered by doing so?
I may regret this later, but to respond to “just an ordinary inventor,” I should say a couple of things in defense of eBay and KSR.
(1) eBay brought the patent law in line with everything else. Injunctions are equitable remedies, imposed according to equitable considerations. The mischief comes from Justice Kennedy’s opinion. If judges are granting injunctions to businesses and refusing them to individuals, simply because the former are businesses and the latter are individuals, that’s a problem with the judges and with Justice Kennedy, not with eBay as such. eBay may have enabled that sort of behavior, but the decision itself seems clearly correct, and it’s up to the judges in the trenches to apply it properly.
(2) KSR–unless you have a good workable rule in mind for what “obviousness” means, or unless you think that standards (as opposed to rules) have no place in patent law, I don’t think you have a right to complain. As with eBay, if judges are issuing result-oriented decisions based on improper considerations, that’s a problem with the judges. If you can think of a good rule that would capture the meaning of “obviousness” and constrain the judges appropriately, feel free to suggest it here.
I agree with Malcolm Mooney about the sampling thing. The Supreme Court divides several times every year in statutory interpretation cases, and they disagree among themselves over things like how plain “plain” meaning is, whether and to what extent “purpose” and legislative history matter, etc. And while some of those statutory interpretation questions have no clear answer, most people agree that most laws are written clearly enough to allow people to conduct their business. This is not to say that the Supreme Court hasn’t fouled up other areas of the law (takings, constitutional limits on punitive damages), only that it would be silly to say that statutory interpretation is “inherently indeterminate.”
So it is with claim construction, I think. Phillips sets out of bunch of guidelines that don’t provide clear answers to every question of claim construction, but allow people to buy, sell, trade and license patents every day. The questions that get into the courts and that get appealed are the tough ones, usually.
Though I also agree with Erik Flom. In some cases, there is more than one reasonable answer, and no principled way to say that one is more “correct” than the other, and under those circumstances it makes more sense to defer to the judgment of the district court. Cybor should be scrapped, whether by Congress, the Supreme Court, or the en banc CAFC.
In any event, thanks to Prof. Schwartz for sharing this. Unlike much of the other commentary out there, this piece should advance the debate considerably.
Dear Mr. David Schwartz,
Your three CLAIM CONSTRUCTION REVERSAL RATE articles are extremely important and I’m sure they are read and understood by the most serious Patently-O readers.
I also suggest readers download your full article:
“Practice Makes Perfect? An Empirical Study of Claim Construction Reversal Rates in Patent Cases.”
Dave, please pardon this re-directing comment:
I believe Mr. Sorbel’s comment and my response thereto may also be interesting to your audience, and I therefore invite your readers follow this link to Mr. Curtis Sorbel’s comment:
link to patentlyo.com
The first endnote (copy below) in my comment to Mr. Sorbel is particularly relevant to the Claim Construction Reversal Rates your articles and paper discuss at some length.
The two Supreme Court decisions in the Endnote fan the flames of big business to litigate, litigate, litigate and beat that patent troll to into non-existence no matter how valuable the troll’s invention is, no matter how strong his patent is.
Endnote 1:
(a) The Supreme Court eBay decision, for example, which stole the power of a patent injunction from self-employed inventors but not from other inventors, was patently inconsistent with our Constitution (a patent without the power to stop infringement is almost useless, akin to a revolver without bullets).
(b) By making patent validity, in essence, “flexible”, the Supreme Court’s KSR decision, invited judges to render result-oriented patent decisions at whim — their own personal whim or big business influence whim or media stirred up whim-of-the-day.
These two decisions are only part of the story – more to come.
On this link, Dennis posted an interesting quote from yesteryear.
Little has changed.
link to patentlyo.com
Patent Troll Tracker, Trade Secrets, and other Bits, No. 16:
Quote of the day (from 1895): “It should also be borne in mind that no property is so uncertain as “patent rights”; no property more speculative in character or held by a more precarious tenure. An applicant who goes into the patent office with claims expanded to correspond with his unbounded faith in the invention, may emerge therefrom with a shriveled parchment which protects only that which any ingenious infringer can evade. Even this may be taken from him by the courts. Indeed, it is only after a patentee has passed successfully the ordeal of judicial interpretation that he can speak with any real certainty as to the scope and character of his invention. Especially is this true of patents on spring-tooth harrows.” E. Bement & Sons v. La Dow (C.C.) 66 Fed. 185 (Circuit Ct. N.Y. 1895)
Dear Anon E. Mouse,
Re: “MM’s right in saying that the claims that make it to trial are a special subset of claims. The parties can’t agree what the claims cover, so much so they spend more $$ to argue in front of a judge.”
With all due respect, I don’t think so.
Remarkably, claim construction seems to have gotten more cloudy after en banc Phillips v AWH. You attorneys don’t give yourselves enough credit. Given the inconsistencies in applying claim construction guidelines, as demonstrated, for example, in the vagaries of one CAFC panel decision to the next, patent attorneys worth their salt can spin almost any claim in any direction. To begin with, language, even claim language, is not a precise science by any means.
It is naive to believe that vague claim language leads to litigation — circumstances beyond the clarity of the claims drive litigants into court. For one example, whether or not to litigate may be decided after the bean counter provides financial analysis.
MM’s right in saying that the claims that make it to trial are a special subset of claims. The parties can’t agree what the claims cover, so much so they spend more $$ to argue in front of a judge. Not surprisingly, judges can’t agree what the claims mean, either. Certainly some of the difficulty is in the claims themselves.
There’s another issue, too. These judges who’ve seen 51-100 cases — are they getting the hardest cases or the cases with the most claims? If there’s any leeway on assigning judges in any district court, do they say “give this one to Harry, he’s seen 6 dozen of them?”
“Reversal” could be on one element of one claim, right? No credit for getting the other eleventy-seven elements/claims construed correctly.
I’m startled to see MM and x6k both striking lucid notes on the same topic.
And really, I concur with Malcolm – the patents that don’t settle before trial provide a skewed sample of less-comprehensible claims.
And with x6k – the applications that do issue as patents probably are a skewed sample of more-comprehensible claims among all the broad gibberish that applicants and attorneys try to push through the PTO. ’cause clearly drafted claims are harder to reject, and easier for the applicant to argue, when there is convincing novelty …? looking for your feedback here, 6k. and btw, i hope someday to put an application through your queue … just stick one of your memorable thoughts in the election of species or the 112 of the non-final …
“other factors that point to opposite conclusions, then it should defer to the trial court.”
Or the examiners during prosecution, because otherwise you’ll just have a crpload of people wanting judgment for their maybe obvious, maybe non-obvious claim, because they’re constantly toeing? the line. This is why I’ve espoused sooo many different times that the obviousness call requires judgment, plain and simple. Sometimes you can build a case so strong that there is no real need for any “judgment”, that’s called building a case under Graham.
“is that most claims are easy to construe for nearly all purposes.”
MM, I think you mean most claims that make it to trial. “MOST claims easy to construe”, if referring to the ind. I cannot reasonably say would be a true assessment of the situation I see every(other)day.
I think overall we can agree that between having two sets of judges working on the case generally helps come to a reasonable conclusion for the case. And that, rather than the mere existence of the higher court, is probably the experiment that was really set into motion. Not if decisions of both courts would be consistent almost always.
“I agree with TJ that some district courts are getting it better than others, but it seems to me that explanations (i) and (ii) are a consequence of (iii)”
I don’t think you understand what (i) means.
Oh, and about the plato v normalists(?), yeah, your discussion only counts the part about litigation, where’s the discussion about how the claims are construed during examination? That’s the important part for me. And frankly, I believe that what you are talking about is very illustrative of a major problem in the PTO, some people try to use the metaphysics claim construction, while other people boil it down very quickly to the example chair in the drawings. I was taught to do the plato way, and now people are trying to get me to do more the normalist way to boost production. Which it does. Though, taking the plato route makes for much easier rejections.
Hello MALCOLM (using Seinfeld’s Newman inflection),
Re: “Using this data to argue that the meaning of claims is “indeterminate” (i.e., not possible to determine with any certainty) is, quite frankly, absurd.”
As Ronnie would say — There you go again – “shallow malo,” just posting whatever poops out of your but hole without bothering to read or think about what’s going on.
Malo, funny you are good at sometimes, but rarely at serious stuff.
[drifting back on stool calling for first Sunday afternoon Old Fashion and bag of Pork Rinds]
A lesson I remember from those learned in quantum mechanics at the University of Chicago is that if the data makes no sense, you are probably asking the wrong question. “The wrong question” is one which is inappropriate to the system. Thus if you measure a digital signal of light at the wrong polarization, you get gibberish because you are asking the wrong question.
Here the conclusion drawn is that district court judges don’t learn. That’s gibberish. These judges manage to have low reversal rates in just about every other area of law. A better explanation is that the judges are given contradictory instructions, and are rolling the dice when balancing these contradictory points. Since the Federal Circuit rebalances these contraditory points de novo and with no deference, formal or informal, the results we see obtain.
If the Federal Circuit cannot explain how to balance literalism versus what people of ordinary skill in the art would think (Chef-America v Lamb-Weston anyone? link to ll.georgetown.edu) or other factors that point to opposite conclusions, then it should defer to the trial court. If the result is to be on mere personal preference, it should be the trial court’s — that is where discretionary decisions are supposed to lie.
Otherwise, district court judges are trying to guess the personal preferences of a panel of the Federal Circuit yet to be selected. A game they cannot win.
“we owe Professor Schwartz (who I should disclose is an old friend) a debt of gratitude for putting this question into sharp focus for us.”
I think that putting “the question” into “sharp focus” would require defining “indeterminate” in this context.
The simple fact that is readily obscured by the statistical dust-kicking and pointy-headed invocations of Plato is that most claims are easy to construe for nearly all purposes. That is why the vast majority of disputes about patents do not end up in litigation, much less in trial before a jury, much much in appeal at the Federal Circuit. As others have noted, there are some serious sampling issues going on here. Using this data to argue that the meaning of claims is “indeterminate” (i.e., not possible to determine with any certainty) is, quite frankly, absurd.
But perhaps there is another meaning for “indeterminate” that I’m not aware of. I’m not the first person in this thread to put the term in quotes.
It seems to me that Professor Schwartz has written a very important paper. The importance of his hypothesis that claim construction may be “indeterminate” is difficult to overstate.
I agree with Professor Schwartz that, without further study, the data are only suggestive that claim construction is “inherently indeterminate”. Yet, I suspect that’s the truth of the matter. At least that’s what one theory predicts.
Under that theory, the central problem lies with how we understand patent claims. What are they? My instinct has been that they are what philosophers refer to as “universals”. And “universals” are very hard to agree about.
Plato thought that universals (not a word that he used) are perfect, unchanging, eternal “forms” or “ideas” that correspond to the things that we encounter in every day life. Think about a common thing like a chair. In Plato’s theory of the forms, chairs have a form that exists on another, higher level of existence. That form would be the perfect, eternal, unchanging chair and all chairs that we encounter in our every day lives are mere imperfect reflections of that chair that exists in the realm of the forms.
Under Plato’s theory, the thing that we are describing in a patent claim for a chair would be the form of a chair. Think about it. The elements of a patent claim may be attempting to do something very like describing one of Plato’s forms. We don’t claim the chair our inventor invented. We claim all chairs universally.
But Plato’s theory has drawn a lot of criticism. The critics, who vary, are often called “nominalists.” Nominalists think that Plato’s theory of the forms is nonsense. After all, where is this higher realm where the forms are located? And are they concrete things, ideas or something else? But mostly the nominalists criticize Plato because they think that you don’t need forms, or complicated levels of being, to explain how we use the word “chair” to refer broadly (or even universally) to chairs. For them, the problem is linguistic, not metaphysic.
They would think about a patent claim very differently than Plato. They would consider a patent claim to be a part of a legal document that sets out a right to exclude. They would think about a claim in terms of the words used and the rules that govern the meaning of those words in the context of the legal document. They would say that a patent claim is about words. Plato would say it’s about things.
Importantly, if claim construction is about things, then our approach to construction should be different than if it is about the words.
Until we decide which it is, we will continue to conflate the two and claim construction will remain indeterminate.
Whether you agree or not, I think the question whether claim construction is indeterminate is hugely important and we owe Professor Schwartz (who I should disclose is an old friend) a debt of gratitude for putting this question into sharp focus for us.
I think an interesting comparison would be with reversals of the BPAI on claim construction. The question would be whether BPAI judges, who are patent law specialists, satisfy the Federal Circuit more than generalist district court judges.
I agree with TJ that some district courts are getting it better than others, but it seems to me that explanations (i) and (ii) are a consequence of (iii)
Wouldn’t the difference across districts cut against the “inherently indeterminable” thesis? Perhaps there is a lot of variance, but it does seem that some judges (or at least some districts) consistently do better than others, in a way that is not entirely random (as “inherently indeterminable” would require).
Comments are closed.