Product Manager II - LLM Evaluations

We are seeking a highly motivated and experienced Product Manager to lead our new Large Language Model Observability product. This is an exciting opportunity to shape the future of observability in a dynamic and innovative environment in a nascent market. 

As the Product Manager for LLM Observability, you will work with engineers and designers to build a product that our customers love and will use everyday to monitor and understand how well their LLM-based applications are working. We are looking for a passionate, hands-on product manager who will dive into conversations with customers, deeply understand their daily workflows and pains and ship a product that is integral to their daily workflows for monitoring and troubleshooting AI/ML model performance. Working with this team is an opportunity to drive impact in the rapidly growing field of AI evaluations and collaborating with the best Applied Scientists in the industry. 

At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them.

What You’ll Do

• Lead the development and execution of the roadmap for a large portion of our LLM Observability product
• Work closely with engineers and designers to define the product feature, functionality as well as requirements for backend systems and user experience
• Run beta programs with customers collecting feedback on the feature, functionality and usability of the core product offering
• Partner with our AI research team and applied scientists to conceptualize, test and productize new ideas
• Breakdown complexity and ambiguity into prioritized set of problems for Engineering to solve
• Responsible for taking this product to the public, including sales and marketing enablement in partnership with marketing teams
• Use your understanding of ML/AI or a relevant technical space understand end user workflows and translate that into product and design requirements

Who You Are

• Expertise in AI/ML either as a practitioner (Data scientist) or a Product Manager 
• 3+ years of PM experience or experience in the ML/AI space
• Working knowledge of ML, data platforms, data engineering (e.g. key LLM/ML frameworks such as Langchain,, CrewAI, pyTorch etc.)
• Comfortable diving into the code and running small LLM experiments when needed!
• Strong, inherent understanding of customer workflows and challenges with MLOps tooling
• A strong inclination towards taking initiative
• Preferred: previous 0 to 1 product experience

Benefits & Growth:

• New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
• Continuous professional development, product training, and career pathing
• Intra-departmental mentor and buddy program for in-house networking
• An inclusive company culture, ability to join our Community Guilds
• Access to Inclusion Talks, our Internal panel discussions
• Free, global Spring Health benefits for employees and dependents age 6+
• Competitive global benefits

Benefits and Growth listed above may vary based on the country of your employment and the nature of your employment with Datadog.