Busigence Technologies Jobs

PySpark Developer

Busigence Technologies

PySpark Developer

Posted 6 Hours Ago

Be an Early Applicant

Remote

Hiring Remotely in USA

Junior

Remote

Hiring Remotely in USA

Junior

Develop and maintain production-grade PySpark data and ML pipelines: ingest from multiple sources, implement distributed algorithms and complex SQL, optimize performance, integrate APIs/streams/files, and collaborate with data science to deploy scalable solutions.

The summary above was generated by AI

This is a remote position.

Please go through the entire job post thoroughly before pressing Apply. Post pressing Apply, you shall reach the assessment page that must be attempted.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Busigence is a Decision Intelligence Company. We create decision intelligence products for real people by combining data, technology, business, and behaviour enabling strengthened decisions.

PySpark Developer

Team: Engineering

Location: Remote

Relevant Exp: 0-4 Years

Background: Been there-Done that

Compensation: Above industry standards

Requirements

This is an immediate requirement. We shall have an accelerated interview process for fast closure - you would required to be proactive and responsive

Remote position (work-from-anywhere)

Immediate joiners must apply

Data Engineering Experienced - course/competitions/internships/job (<4 years)

Competitive compensation

************

| MUST HAVE |

************

1. Code in Python3 - Numpy?

2.Code in Python3 - Pandas?

3.Code in PySpark3 - Core?
4. Code in PySpark3 - SQL?

5.Developed data engineering pipelines on real-world problem (not just toy projects)?

6.Implemented advanced SQL queries

7.Developed complex logics in PySpark3
8.Confidence to learn PySpark3 -MLlib within two weeks?https://spark.apache.org/docs/latest/api/python/reference/pyspark.ml.html (we shall guide but won't spoon-feed)

===========================================

We are offering one of the most challenging & exciting work on Data Pipelines and Machine Learning Pipelines. You shall be working on sophisticated platforms, products and applications

===========================================

We are looking for developer with real passion for data science pipelines. This is a specialist and individual contributor role. Product development experience preferably at a startup or a lean team is desired

ROLE

We are looking for engineers with real passion for distributed computing with actual hands-on experience developing data application on PySpark. You would be required to work with our data science team on development of several data applications.

Mandatory

1. Must be able to fetching data from data sources (databases, APIs, flat files, etc.)

2. Must know in-and-out of functional programming in Python with strong flair for data structures, linear algebra, & algorithms implementation

3. Must be able to convert, break, & distribute existing Python codes to functional programming syntax

4. Must have worked on atleast one real world project in production on PySpark

5. Must have implemented complex mathematical logics through PySpark at scale on parallel/distributed clusters

6. Must be able to recognize code that is more parallel, and less memory constrained, and you must show how to apply best practices to avoid runtime issues and performance bottlenecks

7. Must have worked on high degree of performance tuning, optimization, configuration, & scheduling in PySpark

8. Must have integrated APIs, streams, databases, files (JSON, XML, CSV etc) through PySpark

Preferred

1. Good to have working knowledge vinaigrette of first-class, high order, & pure functions, recurisons, lazy evaluations, and immutable data structures
2. A firm understanding of the underlying mathematics will be needed to adapt modelling techniques to fit the problem space with large data (1M+ records)

3. Good to have worked on PySpark MLlib and PySpark ML

4. Configured Checkpointing and Directed Acyclic Graphs (DAG) on PySpark cluster
5. Worked on development of data platform

Benefits

How to Apply

You should apply online by clicking "Apply Now".

For queries regarding an open position, please write to [email protected]

For more information, visit http://www.busigence.com
Products: http://busigence.com/offering

Careers: http://careers.busigence.com

Research: http://research.busigence.com

Jobs: http://careers.busigence.com

Scaling established startup innovating & disrupting various domains through artificial intelligence. We bring those people onboard who are dedicated to deliver wisdom to humanity by solving the world’s most pressing problems differently thereby significantly impacting thousands of souls, everyday.

We are a deep rooted organization with success story having worked with folks from top tier background (IIT, NSIT, DCE, BITS, IIITs, NITs, IIMs, ISI etc.) maintaining an awesome culture with a common vision to build great data products. In past we have served fifty five customers and presently developing Enterprise AI products. More details at http://busigence.com/offering

We work extensively & intensely on big data, data science, machine learning, deep learning, reinforcement learning, data analytics, natural language processing, cognitive computing, and business intelligence.

We look for talent, not skills

1. You must be [super sharp]

2. You must be [extremely energetic]

3. You must be [honourably honest]

In addition to the regular stuff which every good startup offers – Lots of learning, Food, Parties, Open culture, Flexible working hours, and what not….

We offer you: [Greatest work of life]

You shall be working on our revolutionary products which are pioneer in their respective categories. This is a fact.

We try real hard to hire fun loving crazy folks who are driven by more than a paycheque. You shall be working with creamiest talent on extremely challenging problems at most happening workplace

Similar Jobs

Lineate

Software Engineer

15 Days Ago

In-Office or Remote

Senior level

Software

Design and maintain scalable backend services, optimize data processing pipelines, and collaborate with teams for high-performance applications.

Top Skills: Aws EmrDatabricksElasticsearchHadoopJavaPysparkPythonSolr

Lineate

Software Engineer

12 Days Ago

In-Office or Remote

Senior level

Software

The role involves designing, developing, and maintaining scalable backend services using Java and Python, optimizing data processing pipelines, and contributing to architecture decisions while collaborating with cross-functional teams.

Top Skills: JavaPython

BAE Systems, Inc.

Senior Program Manager

27 Minutes Ago

Remote or Hybrid

122K-208K Annually

Senior level

122K-208K Annually

Senior level

Aerospace • Hardware • Information Technology • Security • Software • Cybersecurity • Defense

Lead large-scale facility modernization and expansion programs ($50M–$300M) end-to-end. Define requirements and estimates, select construction delivery methods, apply Earned Value Management for performance tracking, manage P&L and capital, communicate program health to executives and government customers, and govern vendors and risk to ensure on-time, on-budget delivery.

Top Skills: Earned Value Management (Evm)Integrated Master Schedule (Ims)Ms ProjectPrimavera P6

What you need to know about the San Francisco Tech Scene

San Francisco and the surrounding Bay Area attracts more startup funding than any other region in the world. Home to Stanford University and UC Berkeley, leading VC firms and several of the world’s most valuable companies, the Bay Area is the place to go for anyone looking to make it big in the tech industry. That said, San Francisco has a lot to offer beyond technology thanks to a thriving art and music scene, excellent food and a short drive to several of the country’s most beautiful recreational areas.

Key Facts About San Francisco Tech

Number of Tech Workers: 365,500; 13.9% of overall workforce (2024 CompTIA survey)
Major Tech Employers: Google, Apple, Salesforce, Meta
Key Industries: Artificial intelligence, cloud computing, fintech, consumer technology, software
Funding Landscape: $50.5 billion in venture capital funding in 2024 (Pitchbook)
Notable Investors: Sequoia Capital, Andreessen Horowitz, Bessemer Venture Partners, Greylock Partners, Khosla Ventures, Kleiner Perkins
Research Centers and Universities: Stanford University; University of California, Berkeley; University of San Francisco; Santa Clara University; Ames Research Center; Center for AI Safety; California Institute for Regenerative Medicine