Some careers have more impact than others.
If you’re looking for a career where you can make a real impression, join HSBC and discover how valued you’ll be.
We are currently seeking an experienced professional to join our team in the role of Senior Software Engineer.
Business: Group Data Technology
Principal responsibilities
• Design and Build Data Processing Systems:
• Collaborate with cross-functional teams to understand data requirements and design efficient data pipelines.
• Implement data ingestion, transformation, and enrichment processes using GCP services (such as BigQuery, Dataflow, and Pub/Sub).
• Ensure scalability, reliability, and performance of data processing workflows.
Data Ingestion and Processing:
• Collect and ingest data from various sources (both batch and real-time) into GCP.
• Cleanse, validate, and transform raw data to ensure its quality and consistency.
• Optimize data processing for speed and efficiency.
Data Storage and Management:
• Choose appropriate storage solutions (e.g., Bigtable, Cloud Storage) based on data characteristics and access patterns.
• Create and manage data warehouses, databases, and data lakes.
• Define data retention policies and archival strategies.
Data Preparation for Analysis:
• Prepare data for downstream analytics, reporting, and machine learning.
• Collaborate with data scientists and analysts to understand their requirements.
• Ensure data is accessible, well-organized, and properly documented.
Automation and Monitoring:
• Automate data workflows using tools like Apache Airflow or Cloud Composer.
• Monitor data pipelines, troubleshoot issues, and proactively address bottlenecks.
• Implement alerting mechanisms for data anomalies.
Security and Compliance:
• Apply security best practices to protect sensitive data.
• Ensure compliance with industry regulations (e.g., GDPR, HIPAA) and internal policies.
• Collaborate with security teams to address vulnerabilities.
Documentation and Knowledge Sharing:
• Document data pipelines, architecture, and processes.
• Share knowledge with team members through documentation, training sessions, and workshops.