{"id":"6425117004","title":"Sr. Principal Architect-Data","posted_at":"2026-11-11T15:25:42.000Z","apply_url":"https://lilly.wd115.myworkdayjobs.com/en-US/LLY/job/India-Hyderabad/R4-Data-Engineering_R-107299-1","locations":["Hyderabad, India"],"employment_type":"full_time","workplace_type":null,"seniority_level":"staff","description_language":"en","source_name":"workday","source_url":"https://lilly.wd115.myworkdayjobs.com/en-US/LLY/job/India-Hyderabad/R4-Data-Engineering_R-107299-1","salary":null,"job_summary":null,"job_description":null,"description_text":"At Lilly, the work is demanding because patients are waiting. We unite caring with discovery to help make life better for people around the world, knowing that every decision, every detail, and every day matters. Headquartered in Indianapolis, Indiana, our over 50,000 employees around the globe take on complex challenges to discover and deliver life-changing medicines, strengthen how health is understood and managed, and support the communities we serve. This is hard, urgent, selfless work - but it's work worth doing. If you're driven by purpose and ready to bring your best to work that truly matters for patients, we invite you to join us.\n\n**Principal Architect-R4**\n\n**Databricks AI Design & Architecture | Agentic AI Systems**\n\nAs a Principal Architect for Databricks AI & Agentic Systems, you will own the end-to-end technical strategy and architecture of Lilly's AI data platform. You will design the systems that enable autonomous, multi-agent AI workflows - spanning data ingestion, semantic modeling, model orchestration, and real-time decision intelligence - while setting the architectural standards the engineering organization builds against.\n\nThis is a senior individual contributor and technical leadership role. You will be the definitive technical authority on Databricks architecture, agentic AI design patterns, and AI/ML platform engineering - directly influencing how Lilly uses AI to transform pharmaceutical R&D, commercial operations, and patient care.\n\n**KEY RESPONSIBILITIES**\n\n**AI Architecture & Agentic System Design**\n\n- Architect multi-agent AI systems on Databricks, defining agent orchestration frameworks, tool-use patterns, memory/state management, and inter-agent communication protocols.\n- Design agentic workflows that combine LLMs, retrieval-augmented generation (RAG), and structured data reasoning to support autonomous decision-making pipelines.\n- Establish architecture patterns for agent-to-agent orchestration using frameworks such as LangGraph, AutoGen, CrewAI, or equivalent, integrated within the Databricks ecosystem.\n- Define how AI agents interact with Unity Catalog assets - tables, models, volumes, and functions - as tool surfaces for autonomous data retrieval and action execution.\n- Architect real-time and near-real-time agentic pipelines leveraging Databricks Structured Streaming, Delta Live Tables, and event-driven triggers.\n\n**Databricks Platform Architecture**\n\n- Own the Databricks Lakehouse architecture end-to-end: catalog/schema design, medallion architecture (raw → refined → curated → AI-ready), storage optimization, and multi-workspace governance.\n- Define and enforce Unity Catalog standards for AI assets - model registries, feature tables, vector search indexes, and function-as-a-tool registrations - enabling governed AI consumption across the enterprise.\n- Architect the semantic and metrics layer so business logic is expressed once, consistently consumed by BI tools, AI agents, and downstream APIs.\n- Set Delta Lake standards: table properties, Z-ordering, partitioning strategies, liquid clustering, and cost-based optimization for petabyte-scale workloads.\n- Design Databricks Model Serving architectures, including real-time inference endpoints, batch inference pipelines, and A/B model deployment patterns.\n\n**AI/ML Platform & LLM Integration**\n\n- Define the enterprise architecture for LLM integration on Databricks - covering prompt management, context assembly, vector stores (Databricks Vector Search), embedding pipelines, and LLM fine-tuning workflows.\n- Architect MLflow-based model lifecycle management: experiment tracking, model versioning, champion/challenger patterns, and governance-compliant model promotion.\n- Design feature engineering and feature serving architectures using Databricks Feature Store, ensuring AI agents have low-latency access to curated, consistent features.\n- Evaluate and adopt emerging Databricks AI capabilities - Mosaic AI, AI/BI Genie, DBRX, and Databricks Apps - as first-class architectural components.\n- Define patterns for responsible AI: model monitoring, drift detection, explainability, audit trails, and bias mitigation within regulated pharmaceutical contexts.\n\n**Governance, Security & Data Quality**\n\n- Define and enforce data governance, access control, lineage, and auditing across all Databricks workspaces using Unity Catalog and integration with enterprise identity providers.\n- Architect data quality frameworks (Great Expectations, Databricks DQ rules, or equivalent) embedded into AI pipelines to ensure model inputs meet reliability standards.\n- Define security architecture for AI workloads: network isolation, secret management, encryption-at-rest/in-transit, IP restriction, and compliance with GxP and GDPR requirements.\n- Own data sharing architecture - Delta Sharing, external connectors, API gateway patterns - ensuring downstream AI consumers receive governed, versioned data surfaces.\n\n**Technical Leadership & Standards**\n\n- Own architectural decision records (ADRs), reference architectures, and platform blueprints that engineering teams build against independently.\n- Lead architecture reviews, design critiques, and proof-of-concept evaluations for platform investments and AI product initiatives.\n- Mentor senior data engineers and ML engineers, elevating architectural thinking, system design rigor, and AI engineering best practices across the organization.\n- Influence and guide vendor partners (Databricks, cloud providers, ISVs) on solution direction, ensuring alignment with Lilly's technical and data governance strategy.\n- Represent Tech@Lilly in external architecture forums, Databricks advisory councils, and industry working groups.\n\n**HOW YOU WILL SUCCEED**\n\n- A production-grade agentic AI platform on Databricks that powers autonomous workflows across commercial, R&D, and patient services domains.\n- Architecture standards clear and comprehensive enough that distributed engineering teams execute with confidence within well-defined boundaries.\n- Measurable platform outcomes: query performance benchmarks met, AI agent latency SLOs achieved, data quality scores at or above targets.\n- Governance posture that passes internal audit and supports regulatory submissions in validated GxP environments.\n- Sustained adoption of AI best practices - evidenced by engineers citing and following the architectural standards you define.\n- Recognized as the go-to technical authority by both engineering and senior business stakeholders on Databricks and AI architecture decisions.\n\n**BASIC QUALIFICATIONS**\n\n- Bachelor's degree in Computer Science, Data Engineering, Artificial Intelligence, or equivalent; advanced degree preferred.\n- 10+ years of experience in data engineering, data platform architecture, or AI/ML engineering.\n- 4+ years of hands-on, production-grade Databricks experience: Unity Catalog, Delta Lake, Model Serving, Workflows, Databricks SQL, cluster lifecycle management, and workspace administration.\n- 5+ years designing and delivering AI/ML systems at scale - covering model training, feature engineering, model serving, and monitoring.\n- Demonstrated experience architecting agentic AI systems: multi-agent orchestration, tool-use patterns, LLM integration, RAG pipelines, and autonomous workflow design.\n- Expert-level SQL and PySpark; strong Python for AI/ML pipeline development.\n- Deep knowledge of data modeling, medallion architecture, Delta Lake internals, and data lakehouse design patterns.\n- Proven track record of translating complex business requirements into scalable, governed, production-ready architectures.\n- Experience leading architecture at the principal or distinguished level - setting standards, mentoring engineers, and influencing technical direction organization-wide.\n\n**PREFERRED QUALIFICATIONS**\n\n- Databricks Certified Data Engineer (Professional) and/or Databricks Certified Machine Learning Professional.\n- Experience with agentic orchestration frameworks:\n- Hands-on experience with Mosaic AI, Databricks Apps, AI/BI Genie, and DBRX or similar open-weight LLMs.\n- Domain expertise in pharmaceutical, healthcare, or regulated industries (GxP, GDPR, HIPAA) - including Customer Master, Product Master, HCP/HCO data, and commercial alignment data.\n- Experience with MLOps and LLMOps practices: CI/CD for ML, model monitoring, prompt versioning, evaluation pipelines.\n- Familiarity with infrastructure-as-code (Terraform, Databricks Asset Bundles) and GitOps for platform configuration.\n- Experience with vector databases (Databricks Vector Search, Pinecone, Weaviate) and embedding model management.\n- Understanding of responsible AI principles, model risk management, and AI governance frameworks in enterprise settings.\n\nLilly is dedicated to helping individuals with disabilities to actively engage in the workforce, ensuring equal opportunities when vying for positions. If you require accommodation to submit a resume for a position at Lilly, please complete the accommodation request form ([https://careers.lilly.com/us/en/workplace-accommodation](https://careers.lilly.com/us/en/workplace-accommodation)) for further assistance. Please note this is for individuals to request an accommodation as part of the application process and any other correspondence will not receive a response.\n\nLilly does not discriminate on the basis of age, race, color, religion, gender, sexual orientation, gender identity, gender expression, national origin, protected veteran status, disability or any other legally protected status.\n\n#WeAreLilly","visa_sponsorship":null,"experience_years_min":4,"job_address":null,"job_city":null,"job_state":null,"job_country":"IN","location_lat":17.4180912,"location_lng":78.3424841,"closed_at":null,"closes_at":null,"apply_with_ai_eligible":true,"keywords":["lifecycle management","production-ready","machine learning","design patterns","data engineers","infrastructure","data modeling","schema design","architectures","communication","stakeholders","data quality","architecture","intelligence","distributed","low-latency","performance","fine-tuning","engineering","end-to-end","operations","enterprise","Delta Lake","monitoring","encryption","production","compliance","regulatory","commercial","mentoring","real-time","pipelines","databases","streaming","Terraform","standards","discovery","ecosystem","workflows","regulated","strategy","scalable","security","ai agent","Platform","business","modeling","pipeline","customer","internal","external","evaluate","leading","agentic","latency","design","Mentor","Python","agents","vector","teams","CI/CD","MLOps","Unity","data","APIs","apps","LLMs","REST","SQL","API","RAG","ML","AI","IP"],"company":{"name":"Eli Lilly","slug":"eli-lilly","logo_url":"https://img.logo.dev/lilly.com?token=pk_fWx5G5QrQMm-0Ud8BW3mBg&size=64&format=png","description":"Eli Lilly is a pharmaceutical company that researches, develops, and manufactures human health medications.","website_url":"https://lilly.com","linkedin_url":"https://linkedin.com/company/eli-lilly-and-company","glassdoor_url":"https://www.glassdoor.com/Overview/Working-at-Eli-Lilly-and-Company-EI_IE223.11,32.htm","x_url":"https://x.com/EliLillyandCo","instagram_url":"https://www.instagram.com/elilillyco/","youtube_url":"https://www.youtube.com/user/EliLillyandCompany","github_url":"https://github.com/EliLillyCo","huggingface_url":"https://huggingface.co/organizations/elilillyco","tiktok_url":null,"crunchbase_url":"https://www.crunchbase.com/organization/eli-lilly","facebook_url":"https://www.facebook.com/elilillyandco/","employee_count_range":"10000+","employee_count":50000,"founded_year":1876,"headquarters":{"address":"Lilly Corporate Center, Indianapolis, IN 46285, United States","city":"Indianapolis, IN","country":"US","lat":39.7683331,"lng":-86.1583502},"industry":"healthtech","company_type":"enterprise","total_funding_usd":6500000,"locations":["AE: United Arab Emirates","AU: Australia","Alberta, Canada","Allentown, PA","Alzey, Germany","Anaheim, CA","Asheville, NC","Atlanta, GA","Au:, Australia","Auckland, New Zealand","Australia","Australia, Sydney CBD","Austria","Avondale, AZ","BR: Sao Paulo","Bad Homburg, Germany","Bakersfield, CA","Baltimore, MD","Bao Ding-保定, China","Barcelona, Spain","Bei Jing-北京, China","Belgium, Brussels","Bengaluru, India","Bogota, Colombia","Bologna, Italy","Boston, Indianapolis","Boston, MA","Bracknell, UK or","Brazil","Bronx, NY","Brooklyn, NY","Brownsville, TX","Bulgaria, Sofia","CA: Toronto ON, Canada","CN: Bei Jing","CN: Hang Zhou","CN: Ji Nan","CN: Nan Jing","CN: Shanghai LAL","Ca: Toronto, Canada","Canada","Canada, Western Canada","Cang Zhou-沧州, China","Carolina, PR","Carolina, Puerto Rico","Chandler, AZ","Chang Chun-长春, China","Chang De-常德, China","Chang Sha-长沙, China","Charlottesville, VA","Cheng Du-成都, China","Chennai, India","Chevy Chase, MD","Chicago, IL","China, A Ke Su-阿克苏","China, An Shan-鞍山","China, Bao Tou-包头","China, Bei Jing-北京","China, Bin Zhou-滨州","China, Cang Zhou-沧州","China, Chang Chun-长春","China, Chang Sha-长沙","China, Chen Zhou-郴州","China, Cheng Du-成都","China, Chong Qing-重庆","China, Da Lian-大连","China, Da Zhou-Si Chuan-达州","China, E Er Duo Si-鄂尔多斯","China, Fu Zhou-福州","China, Guang An-广安","China, Guang Zhou-广州","China, Gui Yang-贵阳","China, Ha Er Bing-哈尔滨","China, Han Zhong-汉中","China, Hang Zhou-杭州","China, He Fei-合肥","China, He Ze-菏泽","China, Heng Shui-衡水","China, Hu He Hao Te-呼和浩特","China, Ji Lin-吉林市","China, Ji Nan-济南","China, Jin Hua-金华","China, Kun Ming-昆明","China, Lang Fang-廊坊","China, Liao Cheng-聊城","China, Luo Yang-洛阳","China, Mian Yang-绵阳","China, Nan Chang-南昌","China, Nan Chong-南充","China, Nan Jing-南京","China, Nan Tong-南通","China, Nan Yang-南阳","China, Ning Bo-宁波","China, Pan Jin-盘锦","China, Qing Dao-青岛","China, Qu Zhou-衢州","China, Quan Zhou-泉州","China, Shang Hai-上海","China, Shang Qiu-商丘","China, Shao Yang-邵阳","China, Shen Yang-沈阳","China, Shen Zhen-深圳","China, Shi Jia Zhuang-石家庄","China, Si Ping-四平","China, Sui Hua-绥化","China, Suzhou Manufacturing-苏州工厂","China, Tai Yuan-太原","China, Tai Zhou-台州","China, Tang Shan-唐山","China, Tian Jin-天津","China, Tie Ling-铁岭","China, Tong Hua-通化","China, Urumuqi-乌鲁木齐","China, Wei Fang-潍坊","China, Wen Zhou-温州","China, Wu Hai-乌海","China, Wu Han-武汉","China, Xi An-西安","China, Xi Ning-西宁","China, Xia Men-厦门","China, Xiao Gan-孝感","China, Xin Xiang-新乡","China, Xing Yi-兴义","China, Xu Zhou-徐州","China, Yan Tai-烟台","China, Yin Chuan-银川","China, Yue Yang-岳阳","China, Zhang Jia Gang-张家港","China, Zhao Qing-肇庆","China, Zheng Zhou-郑州","China, Zhong Shan-中山","China, Zhu Hai-珠海","China, Zhu Ma Dian-驻马店","China, Zi Bo-淄博","Chong Qing-重庆, China","Clear Lake, TX","Coeur d'Alene, ID","Colombia, Bogota","Columbus, OH","Concord, NC","Cork, Ireland","Czech Republic, Praha 8","Da Lian-大连, China","Delhi, India","Denmark, Herlev","Denver, CO","ES: Alcobendas","ES: Alcobendas, Spain","Eau Claire, WI","Ee:, Estonia","Fegersheim, France","Finland, Helsinki","Fo Shan-佛山, China","France, Fegersheim","France, Neuilly Sur Seine","Fredericksburg, VA","Fresno, CA","Fu Zhou-福州, China","GB: Arlington Square","GB: Arlington Square, United Kingdom","Gainesville, GA","Germany, Alzey","Germany, Bad Homburg","Guang Zhou-广州, China","Gui Yang-贵阳, China","Guragon, India","Hang Zhou-杭州, China","He Fei-合肥, China","He Ze-菏泽, China","Helsinki, Finland","Hickory, NC","Hopkinsville, KY","Hot Springs, AR","Houston, TX","Hu He Hao Te-呼和浩特, China","Hungary, Budapest","Huntsville, AL","Hyderabad, India","IE: Global Business Solutions - Cork","IE: Global Business Solutions - Cork, Ireland","IE: Kinsale","IE: Limerick","IN: Chennai","IN: Lilly Bengaluru","IN: Lilly Bengaluru, India","IN: Lilly Hyderabad","IN: Lilly Hyderabad, India","IN: Noida","IT: Milano","IT: Sesto Fiorentino","Idaho Falls, ID","India, Bengaluru","India, Hyderabad","Indianapolis, IN","Indianapolis, IN US","Ireland, Cork","Ireland, Kinsale","Ireland, Limerick","Israel, Ra’anana","Istanbul, Turkey","Italy, Milano","Italy, Sesto Fiorentino","Italy, Verona","JP: Kobe","JP: Kobe Manufacturing","JP: Tokyo","Japan, Kobe","Japan, Seishin laboratories","Japan, Tokyo","Jersey City, NJ","Ji Nan-济南, China","Johnstown, PA","Ka Shi-喀什, China","Kai Feng-开封, China","Katwijk, Netherlands","Katy, TX","Kennewick, WA","Kenosha, WI","Kinsale, Ireland","Kobe","Korea, Busan","Korea, Seoul","Kuwait City, Kuwait","Lan Zhou-兰州, China","Lancaster, PA","Latvia, Riga","Lebanon, IN","Lebanon, India","Limerick, Ireland","Lincoln, NE","Lisbon, Portugal","Little Rock, AR","Ljubljana, Slovenia","Longview, TX","Los Angeles, CA","Lv:, Latvia","MA or Indianapolis, IN","MY: GBS Kuala Lumpur","Malaysia, Petaling Jaya","Marquette, MI","Mexico City, Mexico","Mexico, Mexico City","Miami, FL","Milan, Italy","Minneapolis, MN","Missoula, MT","Mobile, AL","Modesto, CA","Monterrey, Mexico","Montgomery, AL","Mumbai, India","Munster, India","NL: Katwijk","NL: Katwijk, Netherlands","NL: LGNH Utrecht","Nagpur, India","Nan Chang-南昌, China","Nan Jing-南京, China","Nan Tong-南通, China","Nan Yang-南阳, China","Nashville, TN","Netherlands, Katwijk","Netherlands, Utrecht","Neuilly Sur Seine, France","New Orleans, LA","New York, NY","New Zealand, Auckland","Newport Beach, CA","Ning Bo-宁波, China","Omaha, NE","Ontario, Canada","Orlando, FL","Pasadena, CA","Pasco, WA","Petaling Jaya, Malaysia","Philadelphia, PA","Pittsburgh, PA","Plainfield, IN","Plattsburgh, NY","Pleasant Prairie, WI","Pocatello, ID","Poland, Warsawa","Portland, OR","Portugal, Lisbon","Poughkeepsie, NY","Praha 8, Czech Republic","Providence, RI","Puerto Rico, Carolina","Qing Dao-青岛, China","Quebec, Canada","Ra’anana, Israel","Redwood City, CA","Richmond, VA","Riga, Latvia","Riyadh, Saudi Arabia","Romania, Bucharest","SA: Saudi Arabia","Salt Lake City, UT","San Antonio, TX","San Carlos Ca","San Diego, CA","San Fernando, CA","San Francisco, CA","Saudi Arabia, Riyadh","Scarborough, Canada","Seishin laboratories, Japan","Sesto Fiorentino, Italy","Shang Hai-上海, China","Shanghai, China","Shen Zhen-深圳, China","Shi Yan-十堰, China","Si:, Slovenia","Singapore","Slovakia, Bratislava","Slovenia, Ljubljana","Solna, Sweden","Spokane, WA","Springfield, MO","Stamford, CT","Sugar Land, TX","Suzhou Manufacturing-苏州工厂, China","Sweden, Solna","Switzerland, Vernier","Sydney CBD, Australia","Synapse, Singapore","São Paulo, Brazil","TR: Istanbul","TW: Taipei","TW: Taipei, Taiwan","Tai An-泰安, China","Tai Yuan-太原, China","Tai Zhou-台州, China","Taipei, Taiwan","Taiwan, Taipei","Tian Jin-天津, China","Toledo, OH","Topeka, KS","Toronto, Canada","Turkey, Ankara","Turkey, Istanbul","Tyler, TX","UK, Basingstoke","US, Boston MA","US, Clifton NJ","US, Concord NC","US, Huntsville AL","US, Indianapolis IN","US, Lebanon IN","US, Los Angeles CA","US, New York NY","US, Pleasant Prairie WI","US, Richmond VA","US, San Diego CA","US, San Fernando CA","US, San Francisco CA","US: Boston MA Centessa","US: Boston MA Lilly Harborside Research & Development Labs","US: Boston MA Lilly Seaport Innovation Center","US: Boston Verve Therapeutics","US: Indianapolis IN Corporate Center","US: Indianapolis IN Parkwood West","US: Indianapolis IN Tech Center North","US: Indianapolis IN Tech Center South","US: Lebanon IN LP1","US: Lebanon IN LP2","US: Lebanon IN Lilly Medicine Foundry","US: Louisville CO Site 3","US: Research Triangle Park NC","US: South San Francisco","US: South San Francisco Haskins","Uinted Arab Emirates, Dubai","Urumuqi-乌鲁木齐, China","Us: Houston, TX","Us: Huntsville, AL","Us: Kenosha County, WI","Us: Lehigh Valley, PA","Us: Research Triangle Park, NC","Us: Richmond, VA","Utrecht, Netherlands","Valley Stream, NY","Venice, Italy","Vernier, Switzerland","Victoria, Australia","Vienna, Austria","Vilnius, Lithuania","Watertown, MA","Wausau, WI","Wei Fang-潍坊, China","West Palm Beach, FL","Wichita, KS","Winston-Salem, NC","Wu Hai-乌海, China","Wu Han-武汉, China","Xi An-西安, China","Xia Men-厦门, China","Xiao Gan-孝感, China","Xin Xiang-新乡, China","Yue Yang-岳阳, China","Zagreb, Croatia","Zheng Zhou-郑州, China","Zhu Hai-珠海, China"]},"visa_sponsorship_source":null,"h1b_filings":null,"can_apply":true,"application":null}