Staff Software Engineer - OpenTelemetry Python instrumentation | UK | Remote

Grafana Labs · United Kingdom (Remote) · GBP 107,000 - GBP 134,000 · remote Grafana Labs is seeking a Staff Software Engineer to focus on OpenTelemetry Python instrumentation, particularly for Generative AI. The successful candidate will work remotely and be part of a global team. Grafana Cloud is a composable observability platform integrating metrics, logs, and traces with Grafana. The Observability department enables developers to understand application and infrastructure health in any environment by providing tools to instrument code, ingest observability data into Grafana Cloud, and visualise it. As a Staff Software Engineer, you will develop improvements to the OpenTelemetry Python SDK and instrumentation libraries, focusing on AI applications and agent workflows. You will collaborate with open-source communities, build Python instrumentation for AI agents, improve Python SDK performance, and promote best practices around OpenTelemetry. Key responsibilities: Develop and maintain OpenTelemetry Python SDK and instrumentation libraries Collaborate with maintainers and other vendors on specifications, APIs, and implementation decisions Build instrumentation for AI agents, focusing on agent runs, model calls, and tool execution Improve Python SDK performance using profiling and benchmarks Promote best practices around OpenTelemetry and teach others Requirements: 8+ years of software development experience with strong Python proficiency Experience debugging production software, language internals, libraries, and asynchronous code Interest in AI applications and agent systems Experience working in open-source communities Nice to have: Experience with OpenTelemetry or Prometheus Python instrumentation techniques, profiling, runtime diagnostics, or asynchronous context propagation Technical leadership experience In terms of compensation, the range for this role in the UK is GBP 107,000 - GBP 134,000. The company offers Restricted Stock Units (RSUs) and a global annual leave policy of 30 days per annum. The ideal candidate will be able to estimate, plan, coordinate, and deliver large tasks spanning multiple systems. They will actively coach and mentor other team members and be able to identify and resolve issues with technology and product processes. Benefits include: 100% remote, global culture Scaling organisation with meaningful work Transparent communication and open decision-making Innovation-driven environment with autonomy and support Open-source roots and community-driven values Empowered teams with a high-trust, low-ego culture Career growth pathways and defined opportunities Approachable leadership and transparent executives Passionate people who care deeply about their work