SAP lets AI play key role in its certification renewal process

SAP professionals can now renew their certifications through AI-based roleplays. The new service builds on SAP’s revamp of its certification program last year, shifting the focus from rote learning of facts towards real-world use cases. “The evolution of Stay Certified reflects a broader shift already taking place across SAP Certification: moving beyond assessing what someone knows toward validating how they can apply that knowledge. AI-powered roleplays bring that principle into the annual certification renewal experience,” the company said in a blog post announcing the new option. The AI-based scenarios across more than 100 certifications simulate customer interactions with a defined knowledge goal. “The scenarios are grounded in content reflecting the latest SAP product releases, helping learners demonstrate applied, up-to-date knowledge,” SAP said. Immediate feedback is provided to highlight what went well during the roleplay, and what needed improvement. However, the use of the AI isn’t mandatory: IT pros can still choose to complete a standalone assessment quiz when renewing their certifications. The process Users have a 90 day renewal window before certification expiry to complete their recertification. To do so, they must have a subscription to the SAP Learning Hub, at $1452 per user per year, which provides expert-led live sessions to help prepare them, access to hands-on practice systems, four SAP certification exam attempts per year and access to “enablement and assessment to keep certifications valid.” Alternatively, SAP said, users can purchase and retake a certification exam each year. Yet another option, being piloted at SAP TechEd in Berlin, will be an event-based approach giving eligible users the ability to fulfill their Stay Certified requirement via on-site, hands-on sessions. A move in the right direction Analysts are guardedly impressed with the revised approach. “I think SAP is moving in the right direction, and the ERP industry could have addressed this sooner,” said Robert Kramer, managing partner at KramerERP. “With cloud, AI and automation advancing at their current pace, traditional certification models are struggling to keep up. Knowing the software is one thing. Understanding how to apply it to a customer’s business is another.” He likes the idea of AI-powered roleplays, because they move certification closer to the application. “Evaluating how someone handles a customer conversation, understands a business problem and recommends a solution is more meaningful than testing whether they memorized product features,” he said. Frank Dickson, principal analyst at Dickson Research, agreed. The quiz-based approach to renewals was inconsistent, he pointed out: “A credential earned by doing should not be maintained by answering trivia. Airline pilots do not stay current by retaking a written test. They go back into the simulator. That is the right model for cloud software that changes constantly.” Valence Howden, an advisory fellow at Infotech, added, “SAP’s revised approach to certification is a positive step and a good idea, although it’s not a new or novel approach. In-room and virtual confirmation of the ability to apply training beyond the exam has existed in the industry for a long time, from Cisco certifications a few decades ago to ITIL and other frameworks where top-level certification created scenarios to be solved and addressed in order to validate the ability to apply learning.” So, he said, “SAP has taken an old but valuable method that will improve the value of their certification, the value of those who achieve it, and provide some assurance around their skills for those who hire them.” Sanchit vit Gogia, principal analyst at Greyhound Research, agreed. “The AI roleplay is not new to SAP certification,” he said. “It arrived in first-time exams a year ago. What is new is that it has reached renewal, across more than a hundred certifications, and renewal is where SAP’s installed base lives.” But, he said, “The deeper question is who comes to own the definition of competence, because SAP supplies the AI that helps the consultant, plays the customer, and assesses the exchange. That is the part history will weigh.” Questions remain While Kramer approves of the model, he still has reservations about the new testing regime. “A successful roleplay doesn’t necessarily make someone a capable consultant,” he said. “ERP implementations are rarely textbook exercises. They involve decades of customer processes, customizations, industry requirements, questionable data and competing business priorities. Sometimes the best recommendation is not to implement the latest capability, but to address the customer’s readiness first.” And, he added, “there’s also the question of governance and accountability. Who decides what counts as a correct response? How are assessments validated? What happens when the AI gets it wrong? These questions matter when professional credentials and customer confidence are at stake.” Dickson, too, has concerns. “The umpire is now an algorithm. Every roleplay is graded by AI, so the strike zone has to be the same for every batter,” he said. “SAP has not said how scoring is calibrated, audited, or appealed. Two consultants giving the same answer should get the same result. Staying certified also requires an SAP Learning Hub subscription, according to SAP’s own FAQ. SAP sells the training and grades the test. That makes transparency on scoring more important, not less.” SAP is right to test whether certified people can still apply what they know, he noted, and the roleplay is a good simulator. “But a simulator still needs a fair, transparent grader,” he said. He also has questions about the testing methodology. “Last fall’s exams allowed candidates to use AI tools,” he said. “The announcement does not say what is permitted during a roleplay. If AI assistance is allowed, the roleplay partly measures how well someone works with an assistant. That may be perfectly fine. SAP should simply say so.” Kramer, too, is looking for more detail. “I support the direction, but I would recommend SAP provide metrics that AI-powered certification produces better professionals, not just a more efficient certification process,” he said. “If it does, this could help close the ERP adoption gap.” This article first appeared on CIO.