About Appen
Appen is a leader in AI enablement for critical tasks such as model improvement, supervision, and evaluation. To do this we leverage our global crowd of over one million skilled contractors, speaking over 180 languages and dialects, representing 130 countries. In addition, we utilize the industry's most advanced AI-assisted data annotation platform to collect and label various types of data like images, text, speech, audio, and video.
Our data is crucial for building and continuously improving the world's most innovative artificial intelligence systems and Appen is already trusted by the world's largest technology companies. Now with the explosion of interest in generative AI, Appen is helping leaders in automotive, financial services, retail, healthcare, and governments the confidence to deploy world-class AI products.
At Appen, we are purpose driven. Our fundamental role in AI is to ensure all models are helpful, honest, and harmless, so we firmly believe in unlocking the power of AI to build a better world. We have a learn-it-all culture that values perspective, growth, and innovation. We are customer-obsessed, action-oriented, and celebrate winning together.
At Appen, we are committed to creating an inclusive and diverse workplace. We are an equal opportunity employer that does not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.
Responsibilities:
* Quality Strategy: Help the Quality Director implement a clear overall LLM quality strategy with scalable processes/tools/systems that allow resource efficient monitoring of decision accuracy and surface overall user experience issues fast and effectively to define appropriate continuous improvement programs..
* Quality Metrics: Passionately tell the story of why Quality matters, build consensus, and inspire others to move fast on fixing problems.
* Team Management: Effectively manage and mentor a diverse team responsible for diverse workflows, Quality channels, and global stakeholders.
* Liaison with Program Management teams: Gather problem statements that enable teams to deliver innovative quality solutions. Drive accountability with team leads and their teams to meet quality performance goals and ensure program standards are met on a week-over-week basis;Assess and prioritize the top-quality issues affecting teams across all decision-making channels.
* Leading Metric Development: Identify and develop internal leading metrics, reporting for project teams to improve quality and delivery performance (prior to Client impact).
* RCA’s: Find creative workarounds for quality issues where our tools are the root cause and a technical solution is not imminent. Manage investigations of escalations for inaccurate decisions, to understand the root cause(s), especially in cases where policies are new, the content is ambiguous, or the decision has an outsized impact if incorrect.
* Collaborate with Rater Community Manager: In support of a ‘Crowd as a Customer’ initiative, driving quality standards across our crowd.
* Continuous Quality Improvement Collaboration with Client Teams: Manage a quality improvement program that delivers iterative improvements to processes across all Client teams.
* Visualization Techniques: Define and develop Client SLA, KPI and CPI metrics, reporting and interactive scorecards.
* Communication: Lead and/or participate in regular team meetings. Where necessary, complete additional tasks as assigned.
Qualifications and Experience:
* 2 to 5 years of qualified experience with LLM’s.
* Minimum of 2 years of experience with Information/Business Systems.
* Experience in managing quality control/quality analysis organization or leading cross-functional quality improvement projects and teams in a dynamic environment.
* 5+ years’ experience in quality with a deep knowledge of risk management.
* 5+ years’ of demonstrated management ability across diverse functions (direct reports, cross functional relationships, contractors, 3rd party vendors. etc.)
* 5+ years’ experience managing teams and mentoring others..
* 2+ years of experience in a senior management role working in a metrics driven setting..
* 2+ years of experience in the AI and data training..
* 2+ years of experience managing larger scale customer engagements..
* 2+ years of experience in a client facing role..
* Proven track record of collaborating with cross-functional groups to produce results.
* Demonstrated ability to perform well in a rapidly changing and extremely global team.
* Strong practical experience with Excel (PivotTable, Charts, Statistical functions) is essential.
* Experience with Tableau/Power BI or other similar tools is a plus.
* Excellent communication skills
* Strong critical thinking and exceptional problem-solving skills
* Proven track record of breaking down complex processes and identifying key pain points to deliver business improvements at the regional or global level.
* Passion for our mission of ensuring a world class support experience for our community.
* Quality certification (e.g. Lean Six Sigma, TQM, etc.) a plus
* Project management certification (PMP, Prince2, etc.) is a plus
Demonstrated experience:
* Facilitating root cause analysis and value stream mapping workshops
* Performing statistical analysis to understand correlation.. .
Required Knowledge, Skills and Abilities:
*The idea candidate will be self-driven, an experienced professional, who has demonstrated ability to lead change collaboratively within different team structures and LLM project types..
* Excellent ability to serve as the liaison between Client and Technology teams delivering solutions.
* Technical/professional expertise is demonstrated through problem solving, applying professional judgement, and competent performance.
* Consultative, collaboration approach and demeanour.
* Motivated self-starter who can establish a course of action for self and others and drive initiatives to completion.
* Flexible, independent, self-motivated; able to deal calmly and professionally with ambiguous data while leading in an environment of constant change.
* Excellent communication skills including an ability to present information clearly and concisely in writing or verbally. Must also be an attentive and careful listener and respond appropriately to others.
* Demonstrated time management and organizational skills with attention to details.
* Excellent analytical skills including ability to proactively identify problems, gather information and set course of action.
* Working across a variety of internal stakeholders.
* Working within and helping establish KPI’s and driving results
* Effective quality risk management.
* Proven skills with building internal and external relationships.
* Effectively communicate and partner with project teams to assess impacts with quick actions to remediate change or unexpected fluctuations in quality..