Databricks Administrator Job at Next Ventures, Mountain View, CA

TUxlb1RSQXJSWXFuZDZPS1FENUVKTlZEclE9PQ==
  • Next Ventures
  • Mountain View, CA

Job Description

Onsite at client location in Mountain View, CA (4 days per week)

Long term contract with possibility of conversion (~3 years)

Average hourly rate of $80/hour depending on experience and interview performance

Next Venture's financial services consulting client is seeking a Databricks Administrator with subject matter expertise in AWS services and products for a long term contract position out of their end client's office in Mountain View, CA.

The end client is just beginning their Databricks journey and is seeking a seasoned administrator to leadin setting up, managing, and securing the Databricks platform within an AWS environment, ensuring it runs smoothly for the entire team.

Responsibilities:

Key Responsibilities:

Databricks & Cloud Platform Administration:

  • Deploy, configure, and maintain Databricks workspaces on AWS, ensuring high availability and security.
  • Manage cluster configurations, autoscaling, and compute resources for optimal performance.
  • Administer Unity Catalog , enforcing governance policies for structured and unstructured data assets.

Security, Compliance & Data Governance:

  • Implement Unity Catalog for centralized access control , enforcing RBAC (Role-Based Access Control), attribute-based policies, and lineage tracking .
  • Define fine-grained permissions for tables, views, and files stored in AWS S3 .
  • Ensure compliance with banking industry regulations (e.g., SOX, PCI-DSS, GDPR ) through data masking, encryption, and audit logging .
  • Manage IAM roles and authentication mechanisms between AWS and Databricks.

Performance Optimization & Monitoring:

  • Optimize Spark job performance and cluster tuning using best practices.
  • Monitor Databricks workspace health with CloudWatch, Datadog, or Splunk .
  • Automate cost control measures through idle cluster termination, autoscaling policies, and query optimizations .

Integration & Automation:

  • Integrate Databricks with AWS services (S3, Glue, Redshift, Kinesis, IAM) and external databases .
  • Automate workspace provisioning and Unity Catalog policies using Terraform or Infrastructure-as-Code (IaC) .
  • Implement CI/CD pipelines for Databricks notebooks, jobs, and configurations.

Collaboration & Support:

  • Work closely with data engineers, cloud architects, security teams, and compliance officers to align Databricks with banking security policies.
  • Troubleshoot platform issues, user access problems, and data pipeline failures .
  • Develop best practices and documentation for Databricks administration and Unity Catalog implementation.

Qualifications & Experience:

  • 5+ years of experience administering Databricks on AWS in an enterprise environment.
  • Hands-on experience with Unity Catalog , including fine-grained permissions, lineage tracking, and governance policies .
  • Strong knowledge of AWS services (S3, Glue, Redshift, IAM, Lambda, Kinesis).
  • Expertise in Spark cluster management, Databricks Jobs, and SQL analytics .
  • Experience in cloud security frameworks and data compliance for regulated industries.
  • Familiarity with Terraform, CloudFormation, or Infrastructure-as-Code for automation.
  • Understanding of networking & VPC configurations for Databricks on AWS.

Preferred Skills:

  • Experience with Databricks Delta Lake, MLflow, and AI/ML workloads .
  • Proficiency in Python, Bash, or PowerShell for scripting and automation.
  • Knowledge of Airflow, Kafka, or other data orchestration tools .
  • Databricks Certified Associate/Professional Administrator certification is a plus.

Job Tags

Hourly pay, Contract work,

Similar Jobs

Office of Public Response

Building Inspector I/II (Structural) Job at Office of Public Response

 ...The Building Inspector I performs technical work inspecting building construction on-site for compliance with approved plans and applicable city and state codes. Under supervision of the Chief Inspector, this position demonstrates strong time management and self-direction... 

Maxion Research

Part-Time Paid Research and Data Entry Clerk (Hiring Immediately) Job at Maxion Research

 ...Choose Us? You have the flexibility to work either from home or in-person, with the option...  ...and services, sharing your thoughts and experiences. What We Need From You: - A working...  ...to learn and develop skills for remote work success. What You'll Enjoy: - Participate... 

AKS Engineering & Forestry

Grants Program Manager Job at AKS Engineering & Forestry

 ...strategy, technical understanding, and you know how to turn great ideas into funded projects, this is the role for you. As a Grants Program Manager , youll play a key role in helping communities access critical funding for infrastructure, planning, and development... 

Michels

Semi Dump Truck Driver Job at Michels

 ...best. Our work improves lives. Find out how a career as a Semi Dump Truck Driver can change yours. As a Semi Dump Truck Driver, your...  ...and learn new tasks What it takes: ~1-3 years of related experience ~ Recent experience with operating a semi dump trailer and/... 

Woodlands Senior Living

Certified Residential Medication Aide (CRMA) - 16 hours per week, nights! Job at Woodlands Senior Living

 ...hours per week, nights! Bridgton, ME ( Direct Care Job Type Part-time Description Do you have a background or interest in healthcare with a soft spot in your heart for helping senior citizens and their families? Do you seek a rewarding career within a...