AI Governance & Test Engineer - AVP

Overview

The AI Governance and Test Engineer plays a critical role in ensuring the responsible and effective deployment of AI solutions. This position is central to upholding regulatory compliance, meticulously monitoring the performance and accuracy of AI models, and conducting rigorous performance testing to validate the reliability and efficiency of AI objects throughout their lifecycle. The successful candidate will bridge the gap between AI development, operational excellence, and robust governance frameworks.

Key Responsibilities

AI Governance & Compliance

Proactively track and manage critical dates related to AI model lifecycles, including limitation expirations, policy decisions (PDs), and scheduled model updates.Prepare, review, and submit all necessary risk artifacts, ensuring strict adherence to internal policies and external regulatory requirements.Coordinate and facilitate review sessions with Model Risk Management (MRM) and other pertinent stakeholders to ensure AI models consistently meet established governance, risk, and compliance standards.

AI Maintenance & Monitoring

Conduct in-depth data analysis of AI model usage patterns, performance metrics, and operational data to identify trends, anomalies, and areas for continuous improvement.Perform detailed manual Subject Matter Expert (SME) reviews of AI object outputs to rigorously assess and confirm their accuracy, relevance, and alignment with defined business objectives.Oversee and coordinate the implementation of updates to AI objects, aiming to minimize disruption while maximizing effectiveness and efficiency.Execute fine-tuning processes for AI objects to optimize their performance, accuracy, and responsiveness based on ongoing monitoring results, feedback loops, and evolving requirements.

AI Performance Testing

Design, develop, and execute comprehensive performance tests for AI objects to evaluate their scalability, responsiveness, stability, and resource utilization under various load or hyper-parameter conditions.Analyze detailed test results to pinpoint performance bottlenecks, potential failure points, and areas for optimization within AI systems.Collaborate closely with AI development teams to implement performance improvements, conduct re-testing, and ensure solutions meet or exceed established performance benchmarks and non-functional requirements.

Qualifications

Proven experience in AI/Machine Learning model development, operations, or testing.Strong understanding of AI/ML lifecycle, concepts, and deployment challenges.Familiarity with risk management, governance, and compliance frameworks, preferably within a regulated industry.Proficiency in data analysis tools and techniques, with the ability to interpret complex datasets.Experience with performance testing methodologies and tools.Excellent analytical, problem-solving, and critical thinking skills.Strong communication, collaboration, and stakeholder management abilities.Ability to work independently and as part of a cross-functional team in a fast-paced environment.

What well provide you

By joining Citi Belfast, you will not only be part of a business casual workplace with a hybrid working model (up to 2 days working at home per week), but also receive a competitive base salary (which is annually reviewed), and enjoy a whole host of additional benefits such as:

27 days annual leave (plus bank holidays)A discretional annual performance related bonusPrivate Medical Care & Life InsuranceEmployee Assistance ProgramPension PlanPaid Parental LeaveSpecial discounts for employees, family, and friendsAccess to an array of learning and development resources

Visit our Global Benefits page to learn more.

Alongside these benefits Citi is committed to ensuring our workplace is where everyone feels comfortable coming to work as their whole self, every day. We want the best talent around the world to be energized to join us, motivated to stay and empowered to thrive.

#LI-Rh5>

------------------------------------------------------

Job Family Group:

Technology

------------------------------------------------------

Job Family:

Applications Development

------------------------------------------------------

Time Type:

Full time

------------------------------------------------------

Most Relevant Skills

Please see the requirements listed above.

------------------------------------------------------

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

------------------------------------------------------

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.View Citis EEO Policy Statement and the Know Your Rights poster.