With growing concerns over major retail data breaches, many have been increasing pressure on retailers to become compliant with the retail industry's security standards—formally called the Payment Card Industry Data Security Standard (PCI DSS). Several states are trying to make such compliance legally required.

But behind PCI is an alarming patchwork of contradictory enforcement, auditors selling the services they're critiquing and frustrated retailers who say they can't jump through infinite hoops forever. It's this combination that is behind some of the low PCI compliance figures released recently.

PCI was designed to formalize what retailers considered to be the best security practices and procedures and to provide a precise, consistent way to get merchants to comply. In practice, however, it has slightly improved security while sharply improving raising retailer frustration.

In general, the sensitive nature of a company's security procedures causes most retail IT executives to shy away from publicly discussing their operations and their plans. But the reason many of these executives are hesitant to vent their PCI frustrations is chiefly because of industry politics and the fact that merchants are constantly negotiating with the companies deciding whether or not they will be PCI compliant.

For this column, several retail IT executives, auditors and others have agreed to speak not-for-attribution—and occasionally on-the-record—about the state of PCI enforcement today. In instances where accusations have been made on background, they have been confirmed by at least three independent sources.

It's been said many times that being PCI compliant does not necessarily translate into being secure. The hoops that retailers have to jump through to achieve compliance have more to do with business purchases and relationships with the overseers than they do with security.

PCI is managed by a group of retailers, banks and credit card associations, but—as a practical matter—it's strongly managed by one company: Visa.

"Visa is definitely leading the charge," said David King, CIO of Regal Cinemas, the nation's largest theater chain with 529 theaters and $2.6 billion in annual revenue. "It's Visa calling people. It's Visa people setting regulations, dictating enforcement."

Said the CIO of another multi-billion retailer: "PCI is nothing but a shell company for Visa."

But below Visa is an army of auditors. But unlike the way publicly-held companies must deal with accountants for financial audits in this day of Sarbanes-Oxley, the auditors here work for private companies that invariably sell security software and hardware.

In other words, the auditors who will decide—with remarkable discretion—whether or not a retailer is given the greenlight for compliance are also selling to that retailer services and products that they can decide will make them compliant. Consider an auditor saying, "Based on what I see here, I can't support your accreditation effort, but if you buy this here list of $9 million of our products and services, that would almost certainly change my mind."

One PCI consultant who asked that his name not be used said it's a very straightforward business deal. "Assessments are low profit activities and rather repetitive. For an assessor to make a higher margin, they need to do other things. Since there is no requirement that prevents this (a la Enron-related rules for accountants), the assessors are going to use the knowledge gained from learning about the problems to 'solve' the problems," the consultant said.

"Some assessors are also selling products for compliance purposes," the PCI consultant said. "I don’t have any evidence that assessors are deliberately manipulating findings to favor the products or services they resell, but the temptation is pretty great. Providing assessment isn’t nearly as high margin as providing compliance. Without clearer rules, it’s logical that some companies will cross the line."

Regal's King said he's seen this before. "This used to be the climate that we all lived with before Enron. (Accounting firms) not only did the audits, they also did taxes and evaluated risks. One division created revenue for another division," King said. "The whole PCI compliance industry is like it used to be before all of that occurred."

Linda Walker is the VP of IT Infrastructure and Security for Dick's Sporting Goods, a chain of more than 300 stores and about $3.1 billion in annual revenue. Walker is also taken aback by how far astray PCI regulation has gone today.

"It amazes me that these auditors are even allowed to sell remediation services," Walker said. "If Visa wants to do the audit, then Visa ought to do the audit."

Gordon Rapkin, the CEO of security services firm Protegrity, agrees with the striking parallels to what the financial accounting world looked like 10 years ago.

"Didn't we learn anything from Enron? Here we have a bunch of assessors who have a catalogue of products to sell," Rapkin said. "You've got an assessor whose job is to tell you what's broken. This conflict of interest is the real issue. It's the big one. (The auditor will say) 'I can fix this and it will pass.' This is just a total conflict. We need assessors who will assess."

Rapkin described a mid-May meeting he had in Europe with a group of representatives from Visa, MasterCard and AmericanExpress, plus a few others. When he complained about the conflict, he described their reaction as, "Yeah, that's true. That's right. And one said, 'We have lots and lots and lots of merchants that need to be assessed and not a lot of people who know how to do it or are willing to do it'" for the low fees that pure assessment can generate.

Rapkin said the credit card executive then said, "I know that it doesn't sound right, but it gets us what we need" if we allow auditors to also sell their security products. "It was quite Machiavellian. The end will justify the means."

That conflict of interest wouldn't be as much of a concern were these auditors not given such broad latitude to interpret the PCI requirements.

The PCI auditing procedure enforcement guide is some 50 pages long and is full of very specific rules that could be interpreted in very different ways, said David Taylor, an auditor who is president of the PCI Security Vendor Alliance. "I could pick 10 items and tell you two or three different ways you can legitimately interpret the testing details."

The fact that many rules are subject to varying interpretation is not what Taylor finds so troubling. It's the attitude that many auditors have that the rules are explicit, when they are often anything but. "There's going to be some variation in interpretation," he said. "That I consider inevitable. What is surprising is how adamant people are about their interpretation of things."

Consider PCI requirement 2.2.4: "Remove all unnecessary functionality, such as scripts, drivers, features, subsystems, file systems, and unnecessary web servers." Taylor's take: "When it says 'all,' that's an absolute thing. But a few words later, think about the very concept of 'unnecessary.' Who determines what is necessary? That's a value judgment. Even though the word 'all' is an absolute, the word 'necessary' forces a judgment."

Or consider requirement 3.6: "Fully document and implement all key management processes and procedures for keys used for
encryption of cardholder data." Again, that sounds explicit and specific, but what precisely consistutes something being fully documented? Said Taylor: "Guidance could be a couple of lines or 20 pages."

The biggest Pandora's settop box with PCI today is compensating controls. The original rationale for compensating controls is that retailers have very different environments and that some requirements may not make sense for them. Therefore, the theory continues, PCI will permit a retailer to do some alternative method if that retailer can prove to the auditor that it has come up with a way that is as secure as what the PCI spec dictates.

Protegrity's Rapkin sees compensating controls today being used as a way for some retailers to get out of abiding by the rules because it's too expensive or too difficult. "It's a 'Get Out Of Jail Free' card," he said. "Compensating controls today are at the whim of the assessor."

Rapkin admits that his dislike for compensating controls stems from the fact that they are often used to avoid encryption and his company happens to sell encryption.

Another argument for compensating controls is that they are a compromise and, the theory goes, the only viable alternative to further the goal of improving security. An analogy is the U.S. food pyramid. Nutritionists working on the latest government pyramid wanted to push beef products into the same category as candy—and to move whole grain products into its own category, away from white bread and white rice. But the government argued that many Americans would likely ignore such an extreme pyramid, thereby supporting the position that an adhered to but compromised pyramid would improve diets more than an ideal but ignored one. (OK, so cattle lobbyists also played a role, but let's not go there. It ruins the analogy.)

The rationale behind compensating controls is that a strict adherence to the rules would cause a lot of retailers to stop trying. So in theory, a compromised—but realistic--security plan would make systems safer than an ideal but less-used plan.

The inconsistency wrecks havoc on the plans of retailers. One CIO said that his PCI auditor just resigned and that they are begging the audit firm to force the new auditor to stick with the decisions made by the old auditor, rather than subject the retailer to starting over.

One requirement that was mentioned by two CIOs—which is not explicitly referenced in the rules—is for closed-circuit video cameras to be installed for every Point-of-Sale terminal in the chain. The intent is to make it more difficult for intruders to install devices onto the POS readers to steal credit card data. The only reference in PCI is a vague requirement in 9.1.1 to "use cameras to monitor sensitive areas."

The CIO of one chain calculated that such a move would—on its own—"vaporize north of $10 million, $15 million easily. And then there's an ongoing $2 million to $3 million a year to maintain it. That's $2 million to $3 million for life and it's only going to go up. And then there's inflation."

That CIO continued: "What about storing the images? You have to have someone to monitor those. No one could possibly afford that. This is ridiculous. We can't do business like that. Then they (auditors) ask, 'Do you want to take credit cards or not?' Absolutely asinine."

What was behind that particular demand? A concern about a POS system encryption procedure. "As soon as a credit card is swiped, we immediately encrypt it. But there's a fleeting moment—a micro nanosecond—between when the swipe has occurred and when it's encrypted, between the POS and the magnetic swipe reader." The alternative to the cameras was a higher-end POS system that the auditor's firm happened to sell.

Another common criticism of the PCI program is retailer confusion. Mostly, that confusion involves whether or not they are compliant. There are several reasons for this confusion.

Some larger chains have separate compliance efforts for different groups, so the CIO may not be certain which parts of his chain are compliant.

But a more common issue is timing.

Let's say that a retailer eventually gets a compliance letter on October 1, declaring his chain PCI compliant. That letter doesn't say the chain is PCI compliant for one year, as a driver's license might. Indeed, it doesn't even technically say the chain was compliant as of Oct. 1, but more likely means that the chain was compliant as of the date of the last completed full audit, which was likely several months earlier.

So when that retailer’s CIO is asked, "Are you PCI compliant?" it’s not as easy as saying yes or no. He knows, for instance, that some auditors looking at systems now have different expectations than the auditors who examined the same systems six months ago.. He also knows that some systems have changed.

This is an all too common situation. In fact, several CIOs interviewed for this piece were honestly not sure whether they were truly compliant, which is frustrating.

"We are deep into quite a number of different initiatives in order to become compliant right now," said one Fortune 500 retail CIO. "But it's a changing landscape, a changing process. No matter what happens, we're going to be doing a lot of positioning and then negotiating back and forth with the auditor or whoever will ultimately be certifying it."

Walker, from Dick's Sporting Goods, added that one historical problem with PCI was "finger pointing. Visa did not want to take responsibility to tell the merchant that they're compliant and the acquiring banks did not want to take the responsibility to do it," Walker said. "You don't want vague assurances of compliance or likely compliance. I want the letter for the wall. A letter from somebody saying I'm compliant or not compliant."

One CIO detailed how his chain suffered under the whims of different auditors. This chain had an audit in October 2005 and was given a compliance confirmation in Feb. '06. "Then the rules started changing." For example, the PCI rules required an incident response plan.

"The first year, just having a plan in writing was sufficient. The second year, the plan was scrutinized for content and we were told much more content was required," the CIO said. "For example, they wanted phone numbers for contacts such as the banks and law enforcement agencies. The criteria of how you get graded kind of shifted and things changed from auditor to auditor even in the same company and certainly from auditing company to auditing company."

The same CIO cited encryption as another example. The chain segmented sensitive transaction data using strict network controls. "In 2005, this was accepted. In 2006, it was not. We are now implementing encryption. Despite an official PCI position that compensating controls are permitted, it seems as though our auditors now will no longer accept any compensating controls for encryption."

Another issue with that chain involves something called a Report on Compliance (ROC), which is a form filled out by a retailer that is trying to get a compliance certification. "In 2005, if you were far enough along in a requirement and had a reasonable plan acceptable to the bank, you were considered compliant. They were then going to monitor your progress to the plan. Now you would be considered not in compliance if the plan is not completed."

The retail also had issues with Web logging, where one auditor found logs acceptable and the next auditor insisted on much more extensive--monthly, weekly and daily—logs.

Some retailers that are not Tier One merchants are filing self-audits, where the retailer's own personnel fills out the forms. Walker, of Dick's Sporting Goods, said she has a real problem with PCI self-audits. "I wouldn't ever be comfortable with a self-audit at Dick's. All companies in all tiers should have an external audit performed," Walker said. "Smaller companies don’t even have the in-house expertise to do a self-audit."

Despite these and other problems, PCI has indeed improved retail security. Very few in retail doubt that it has.

One CIO for a large retail chain said the requirements of PCI helped him purchase pieces of equipment—such as high-end routers--that also helped modernize non-security operations.

"To give Visa some credit, they did shake things up and it's definitely improved retail security," the IT exec said. "I can't imagine what would have happened had I asked for $4 million dollars for security two years ago without the hammer of compliance. They would have looked at me like I had two heads."