Pulling Docker images from private registries for containerised applications presents a security challenge. It requires authentication management, network access, and trust across distributed systems. Credentials must be securely handled and rotated, and image pulls can break due to network restrictions or expired tokens. All of this makes deployment and security harder.

To address this, you can use OpenID Connect (OIDC) when pulling Docker images from Cloudsmith into Kubernetes. OIDC adds a layer to OAuth 2.0 that identifies who is making the request. Cloudsmith can then verify that the requests come from a trusted source and can grant short-lived access tokens. So you no longer have to rely on long-lived credentials, enhancing security and simplifying credential management.

Kubernetes clusters often need to pull Docker images from private registries like Cloudsmith. Traditionally, this meant storing long-lived credentials in the cluster, which has some security risks:

Credentials could be compromised if the cluster is breached.

Regular rotation of credentials becomes a manual, error-prone process.

It's hard to apply the principle of least privilege effectively.

Before we dive into the solution, let's review what an Image Pull Secret is in Kubernetes.

An Image Pull Secret is a way to pass credentials to the kubelet to authenticate with a private container registry. Typically, the secret contains:

While Image Pull Secrets solves the immediate authentication problem, using them with long-lived credentials still exposes us to the security risks mentioned earlier. The credentials don’t automatically expire, leaving them vulnerable to misuse if leaked or unrotated.

OpenID Connect (OIDC) allows us to use short-lived tokens instead of long-lived credentials, reducing risk exposure. Here's how it works in the context of Kubernetes and Cloudsmith:

Kubernetes has its own identity in the form of a service account. The service account has an associated JWT (JSON Web Token). During runtime, the following steps are taken:

Kubernetes issues the signed OIDC token (a JWT).

Cloudsmith receives and verifies the JWT and exchanges it for a short-lived Cloudsmith token.

Kubernetes uses the short-lived Cloudsmith token to create or update an Image Pull Secret.

At runtime, the pod requests to pull an image.

Kubernetes pulls an image from Cloudsmith using the Image Pull Secret for authentication.

To automate the process, we use a Kubernetes CronJob. This job runs at regular intervals and performs the following tasks:

Retrieves the Kubernetes service account token.

Exchange service account token for a Cloudsmith token via OIDC.

Create or update an Image Pull Secret with the new Cloudsmith token.

Here's an example of what this CronJob might look like:

2. Gets the Kubernetes service account token.

4. Creates or updates an Image Pull Secret named cloudsmith-pull-secret.

Implementing this OIDC-based solution offers significant benefits:

: The CronJob ensures the token is regularly refreshed, maintaining a good security posture without manual intervention.

: There's no need to manually update credentials, reducing operational overhead.

: The token only has the permissions it needs for a limited time, aligning with security best practices.

Once the Image Pull Secret is created or updated by the CronJob, you can use it in your pod specifications like this:

This pod will use the cloudsmith-pull-secret to authenticate with Cloudsmith and pull the specified image.

Using OIDC to pull Docker images from Cloudsmith into Kubernetes eliminates the need for long-lived credentials. It reduces the risk of leaked credentials, removes the need for manual credential rotation, and aligns with current security best practices.

Authentication in Kubernetes is evolving with significant recent enhancements. Kubernetes v1.20 saw the introduction of 

. These allow the kubelet to dynamically request credentials for a container registry instead of storing static credentials on disk. This was enhanced in v1.33 to support automatic use of the service account token for image pulls. This enables short-lived authentication without needing to manage Image Pull Secrets and reduces complexity. Cloudsmith remains vigilant, and these updates help with our goal of increasing automation in supply chain security.

Ian Taylor

Tech Content Director

Ian Duffy

Site Reliability Engineer

Secure Docker Image Pulls from Cloudsmith to Kubernetes using OIDC

In a world where digital threats lurk at every corner, securing your software supply chain is vital. Without proper tooling, you can’t adequately protect it.

Over the past few months, software supply chain attacks have compromised thousands of organizations. Okta, users of Progress Software's MOVEit file transfer app, and JetBrains’ TeamCity On-Premise development platform are just a few. In 2020, the notorious 

 infected more than 18,000 customers, including Microsoft, Intel, and roughly 100 other companies, as well as a dozen U.S. government agencies.

research suggests the financial impact of supply chain attacks is rising by 15% annually. They predict that the cost to businesses will reach $60 billion by 2025 and $138 billion by 2031. This alarming trend underscores the urgency for robust security measures and regular audits to safeguard against these evolving threats.

Software supply chain security involves protecting all aspects of the software development and deployment process. It's not just about the code you write, but also about the third-party components you integrate. Common threats include 

inadequate control over third-party libraries

. Open source software has accelerated the speed of development and delivery, and is present in an estimated 96% of computer software, according to the 

Open Source Security and Risk Analysis Report 2023.

 Every element, from source code to deployment, is a potential vulnerability point.

Ignoring these risks can lead to catastrophic data breaches and operational disruptions.

How can I mitigate my risk of exposure to a software supply chain attack?

A thorough security audit is essential for identifying and mitigating risks in your software supply chain. So where do you start?

1. Build an inventory of your software supply chain

Map your entire software supply chain to understand where vulnerabilities may exist. You need to list all of your software components, as each presents a potential risk. This includes the code your team writes and all the third party libraries, frameworks, and tools you use to build and deploy your software. Don’t forget about database systems, orchestration tools, and cloud services. For each component, understand its:

Without the right tooling this can be difficult, especially if you have multiple teams that are on different technology stacks or using separate DevOps tools. You need a fully-managed software supply chain product like 

 to know what artifacts your teams are using and where the potential risks lie.

Now that you have mapped each component in your software supply chain, you need to assess its security posture against known vulnerabilities and industry standards. Tools like 

, a free tool for analyzing the quality and security of OSS packages, cangive you insights into the quality and trustworthiness of open-source packages.In addition, standards like the 

 from OWASP can help you determine your dependencies and check them for publicly disclosed vulnerabilities.

Once you’ve identified potential vulnerabilities in your chain, it's time to formulate a plan to address identified vulnerabilities, including updating or replacing insecure components.This may include patching software, updating systems, changing code, or even replacing certain tools or components. It’s crucial to have a detailed action plan for each type of vulnerability.

Continuously monitor to protect your supply chain:

Your software supply chain is not static. Every day your development teams and the global open-source community make software changes that could impact your security. New vulnerabilities are being discovered and new threats emerge. To maintain security in your software supply chain, you must automate ongoing surveillance of the supply chain while enabling your developers to continue innovating at pace.

Use Cloudsmith to streamline your audit process and secure your software supply chain.

Build your software supply chain inventory

 by creating a single source of truth for all your software assets with Cloudsmith. We provide one home for all your components, supporting all of your programming languages, teams, and development pipelines. Each and every package pulled into Cloudsmith is fully traceable, giving you insight into what software was built using what dependencies and by which teams and developers.

. Artifacts coming into your Cloudsmith repositories are subject to the policies and controls that you configure. SBOMs contain all of the components and dependencies, and Cloudsmith ensures the authenticity and origin of your artifacts by scanning your packages for malware and CVE.

 Cloudsmith makes it easy to establish rules-based protocols for handling low, medium, and critical software vulnerabilities so that risks are stopped and quarantined before they can do damage.

Stay in control of your software supply chain

. Build usage policies, pin dependencies, segregate environments, and use package promotion workflows to ensure the right teams, people, and systems have access to use the packages that meet your specific needs.

Continuously monitor and protect your software supply chain.

 Cloudsmith is cloud-based, offering enhanced speed, security, and ease of use. Our multi-tenant, global infrastructure means you can rely on the platform to scale to meet your needs and not worry about troublesome upgrades that lead to lost productivity and developer frustration.

Mastering software supply chain security audits is not just a matter of implementing a set of procedures; it's a continuous commitment to safeguarding your digital ecosystem. As we've seen with high-profile breaches and the rise in attack frequency and sophistication, no organization can afford to overlook this crucial aspect of cybersecurity.

By maintaining an inventory of your software supply chain, vigilantly assessing vulnerabilities, promptly addressing them, and continuously monitoring for new threats, you can create a robust defence against the ever-evolving landscape of digital risks.

Cloudsmith plays a pivotal role in securing the software supply chains of diverse and global clients like PagerDuty, Stack Overflow, FontAwesome, Shopify, and RabbitMQ. With Cloudsmith, the fast, modern artifact management platform, you can ensure that your software supply chain remains secure, resilient, and ahead of the curve in cybersecurity. 

Claire Speedy

Senior Marketing Manager

How to Audit Your Software Supply Chain Security

Observability is emerging as critical DevOps practice to predict, prevent, identify, and address issues in the distributed, dynamic, complex, and constantly evolving applications and infrastructures that characterize modern software systems. Wisely, many teams are adopting observability tools like Datadog, Elastic, and AWS Athena to aggregate logs from their build tooling and better visualize and understand the state of their entire system.

But, observability tools are only as good as the logs they consume.

A high-caliber artifact management solution should be a rich, central source of information about your software supply chain; after all, it hosts your software and your dependencies. This is why we invest heavily in the quality of Cloudsmith’s logs and make sure they’re easy for you to extract your logs and integrate into your observability tools.

Sharpening your vision with Cloudsmith logs

Our logs give you granular visibility into your entire lifecycle of artifacts, user interactions, and system behavior so you can proactively monitor, debug, and optimize your software supply chain and operations. We provide these three main types of logs:

Reduce risk and react faster with audit logs

 audit logs are essential for robust cybersecurity and compliance strategies. Organizational audit logs capture events across your organization, including the creation and deletion of repositories and modifications to settings.

Repository audit logs offer a timeline of non-package actions within a repo (typically administrative activities), like modifying retention rules or creating an entitlement token.

Together, they offer a comprehensive view, and analyzing these detailed records can be extremely useful for the following:

Identify and respond to unauthorized access attempts or suspicious activities in the audit logs.

Trigger alerts for any unusual patterns or anomalies in user activities that indicate a potential security threat.

Track changes in repository settings or retention rules to ensure compliance with organizational policies.

Monitor and audit user actions to meet compliance requirements and regulatory standards.

Investigate and troubleshoot specific issues reported by users or identified through audit logs.

Utilize historical audit logs to trace changes and events leading up to an issue and rectify it quickly.

Identify and resolve issues promptly by reviewing audit logs for any operational discrepancies or errors.

Use audit logs to track repository creation/deletion and help in capacity planning.

Improve OpEx and product planning with client logs

 help you understand how your builds are being consumed by providing a record of every package downloaded by your users, along with information such as date and time, user agent, and IP address.

You can leverage these logs to improve product and operational decisions, for example:

Analyze client logs to understand how packages are being consumed to help you optimize infrastructure and resource allocation.

Monitor package downloads and associated bandwidth usage.

Implement alerts based on download thresholds to proactively manage and optimize cloud costs.

Analyze download trends to align resource provisioning with actual usage, preventing unnecessary expenditure and optimizing your cloud bill.

Use client logs to identify popular packages, enabling prioritization in product development efforts.

Analyze download trends by country and usage by package versions to inform feature improvements.

Anticipate capacity changes with package logs

 provide a detailed record of all package-related events within a repository.

These include unique event IDs, package IDs, event types, dates/times, user IP addresses, and user account details, and they’re useful for:

Analyze package logs to understand resource utilization trends and plan infrastructure scaling accordingly.

 Identify patterns in package downloads, helping you understand peak usage times and plan for increased capacity during high-demand periods.

 Analyze user locations from IP addresses, allowing you to anticipate regional variations in demand and optimize resources accordingly.

 Identify frequently downloaded packages, enabling proactive scaling for high-demand software.

Compatible for convenient extraction and analysis

You can view all of our logs in your Cloudsmith UI, and we provide help documentation for our

. But you can also easily export and integrate them with your observability tools.

access real-time audit logs via the Cloudsmith API for your 

Most observability tools can consume logs from an AWS S3 bucket, either by ingesting data from custom RESTful API or through integrations.

Datadog is a cloud-based monitoring and analytics platform that can 

, providing insights into system performance. The 

, ingests audit logs information, incorporates other status details from Cloudsmith, and displays them on a Datadog dashboard.

Part of the Elastic Stack, Elasticsearch is a distributed search and analytics engine that can 

 consumed from an S3 bucket. Alternatively, 

can be used for collecting custom HTTPJSON events

Splunk is a widely-used platform for searching, monitoring, and analyzing machine-generated data, including logs. You can 

use Splunk to ingest Cloudsmith logs from the AWS S3 bucket

A serverless query service, Amazon Athena allows you to analyze data directly in an Amazon S3 bucket using standard SQL. We have documentation on 

how to use AWS Athena to extract insights from our Client logs

Cloudsmith logs are crucial for enhancing security, compliance, and operational efficiency in your software supply chain. 

 to empower your organization with actionable insights. Or 

 with any questions. We’re more than happy to help.

Ciara Carey

Solutions Engineer

Improving Observability With Cloudsmith Logs

Open-source software (OSS) fuels innovation. Over 96% of commercial applications rely on at least one OSS component (Synopsys, 2023).

At Cloudsmith, we champion OSS and understand its indispensable role in today's software landscape. However, the escalating threat of supply chain attacks targeting OSS demands a robust defence.

Step into our blog series on how to securely consume OSS. Over the next 3 articles, we'll guide you through

OpenSSF's Secure Supply Chain Consumption Framework (S2C2F) which outlines and defines how to securely consume OSS dependencies into the developer’s workflow.

Today, I'll give an overview of some of the most common types of attacks from consuming OSS. These attacks can lead to broken builds, security breaches, unauthorized access, and data leaks.

Attackers are increasingly focusing on OSS, exploiting vulnerabilities to get malicious code into production systems. To achieve this, malicious actors employ three main avenues of attack when it comes to your OSS consumption:

 Publishing bad packages to public indexes like PyPI, npm, or Maven Central.

accounts can be hijacked, projects abandoned, deleted or maintainers can introduce malicious code themselves.

: Risks linked to vulnerabilities in your OSS dependencies, such as Log4Shell, Heartbleed and the recent high-severity vulnerability found in curl.

Languages with community-based package management, like npm for Javascript and PyPI for Python, make publishing and consuming packages easy. One downside to that is that hackers can publish malicious packages to these repositories to trick developers into installing the wrong package.

 found 7,300 malicious OSS packages were discovered across all major package manager registries in 2022 alone.

Let’s have a look at these types of attacks.

Typosquatting is when bad actors upload compromised packages with names similar to popular, existing packages. If a developer makes a typo when including a package name, the malicious but similarly named package is then downloaded and installed instead. This type of attack is tough to detect and can lead to malware execution.

An example of a typosquatting campaign unfolded in early 2023 when thousands of 

malicious Python packages were discovered

 on the public Python Package Index (PyPI). Similar campaigns have been observed in other popular package ecosystems, including:

: The npm platform witnessed a typosquatting campaign leveraging open-source tools, emphasizing the widespread nature of this threat.

: The distribution of SeroXen RAT via a malicious NuGet package raised alarms within the NuGet and Powershell communities.

: Evidence of a malware attack on Rust developers via typosquatting was identified on crates.io.

: Malicious typosquatting malware was uncovered within the Composer repository, underscoring the vulnerability of various ecosystems to such subversive tactics.

: A notable incident involving the exploitation of Kubernetes RBAC via a typosquatting attack on Docker served as a reminder of the far-reaching consequences of this practice.

The prevalence of typosquatting across multiple OSS ecosystems underscores the need for heightened awareness and vigilant measures within the developer and repository communities to effectively detect and mitigate this elusive threat.

Security researcher Alex Birsan found he was able to get “malicious” code (actually just an innocuous test) into software built by more than 35 major companies (He had advance permission from each company to try.) These companies all 

were vulnerable to the injection of malicious packages via public sources

 into their software supply chains. His test demonstrated that the 

 can occur when attackers find out or guess package names in your organization's private repository and pull in the public package instead of the package hosted on the private repository.

. PyTorch had a dependency on a library called torchtriton that was hosted on PyTorch’s private repository. The attacker decided to register a malicous package on the public PyPI repository with the torchtriton name. When a build was triggered pip looked at the public PyPI first and pulled in the malicious package.

What would happen if a bad actor managed to hijack a Maintainer account? Or how about a malicious maintainer intentionally introducing bugs? Yes, sadly, both of those scenarios have happened.

These maintainer-based threats are more likely to happen in projects that have very few maintainers, which is actually the case for many packages in public repositories like npm and PyPI. Let’s look at a few examples.

In 2021, crypto-mining, password-stealing 

 named ua-parser-js. "I believe someone was hijacking my npm account and published some compromised packages (0.7.29, 0.8.0, 1.0.0) which will probably install malware," said Faisal Salman, author of the UAParser.js library. He turned out to be right.

Another npm package, ESLint, had a similar attack in 2018. The attacker gained unauthorized access to the library and inserted code that could have potentially allowed for further malicious activities. These incidents highlighted the need for switching on 2FA for maintainers of popular packages.

A variation on this attack occurred in 2018 with a popular JavaScript library for Node.js called EventStream. The package was 

 that enabled digital attackers to empty Bitcoin wallets. This attack happened when a malicious party used social engineering to be added as a maintainer. The attacker then added a dependency containing malicious code called flatmap-stream. Everyone who used event-stream with the malicious dependency became a potential target.

Although very rare, it is not unheard of for legitimate maintainers to intentionally introduce bugs or delete code in an update for political or other reasons.

 by its maintainer, Brandon Novaki Miller, to protest the ongoing war in Ukraine. He released an update to npm and GitHub that sent peace messages to all users and deleted files belonging to users based in Russia and Belarus.

 were corrupted by their Maintainer, Mark Squires. He intentionally introduced an infinite loop that broke thousands of projects that depend on colors and faker to protest Fortune 500 companies using OSS for free, as well as to highlight his political interests. Mark was locked out of his GitHub accounts, but eventually, he took down his popular faker package altogether.

Another incident of this type occurred in 2016 on npm when 

 removed over 250 of his modules from npm after a disagreement. One of those dependencies was a popular package called left-pad. Laurie Voss, CTO and co-founder of npm, took the unprecedented step of restoring the unpublished left-pad 0.0.3 package.

. The original maintainer may move on or lose interest, and nobody else steps in. These abandoned projects pose a security risk as vulnerabilities go unpatched.

Vulnerabilities in OSS represent an enduring and significant threat. Known vulnerabilities are publicly disclosed weaknesses cataloged in databases like the 

, and are susceptible to exploitation by attackers.

These vulnerabilities can range in severity, with critical ones often enabling Remote Code Execution (RCE) that grants unauthorized control over systems.

At first glance, it might seem easy- if you have an OSS dependency with a known vulnerability in your software, just update it to make the issue go away. But in reality, this can be tricky because:

There's no consistency with how developers are bringing in OSS which leads to security tool blind spots.

You might not be aware that you're using a vulnerable package, particularly when that package is a dependency of another package— commonly known as a transient dependency.

The vulnerable package is a dependency in legacy software that isn’t receiving updates.

The dependency is abandoned OSS. In such cases, nobody is around to fix those vulnerabilities and/or update vulnerable dependencies.

In a nutshell, fixing vulnerabilities in your software isn't always as simple as it sounds, especially when you bring in OSS in different ways, they're buried deep in your software, tied to older programs, or linked to abandoned parts that nobody's looking after.

Let’s take a closer look at the infamous vulnerability Log4Shell to illustrate the threat of vulnerabilities in the OSS you consume.

 was discovered within the widely-used OSS logging tool Log4J, leaving its users and their organizations susceptible to remote code execution attacks.

Log4Shell had an extraordinarily broad and damaging impact:

It had a huge attack surface, including Cloudflare, iCloud, Minecraft servers, Steam, Tencent QQ, and millions more.

 (CVSS) score of 10 (the highest, or worst, possible score) - a successful exploit left organizations wide open to remote code execution attacks.

It was hard to detect because it was typically pulled in as a dependency of a dependency.

customers who had completely mitigated Log4Shell from their environments saw it reenter their systems

 through a continuous integration (CI) process or the installation of a vulnerable third-party application.

Secure Supply Chain Consumption Framework

Attacks targeting OSS are on the rise because they work, and because so many software organizations aren’t taking adequate precautions.

OpenSSF, the Open Source Security Foundation

, unites OSS maintainers, industry leaders, governments, OSS consumers, and more to enhance OSS security for all.

Secure Supply Chain Consumption Framework (S2C2F)

 provides essential guidelines and best practices to bolster security when consuming OSS.

In our next blog, in our series on how to securely consume OSS, we'll delve into the S2C2 Framework, exploring how it can significantly mitigate the risk of OSS attacks.

The Dangers Lurking in Open Source Software

Are your CI/CD pipelines at risk? They might be if you use long-lived, static credentials and tokens. Long-lived, static credentials and tokens are one of the most common causes of 

CI/CD tools need access to cloud services to publish artifacts, deploy software, and access resources on their cloud provider. So, they need credentials. It's tempting to hard-code them.

 left lots of organizations scrambling to rotate and disable API keys they were using to run their builds.

What's the solution? OpenID Connect! OIDC tokens are a more secure way to handle authentication than long-lived credentials, and they remove the need to store your credentials in your CI/CD platform.

 natively. You can use OIDC tokens to authenticate against our API, CLI and our users can even authenticate using OIDC with format-specific endpoints like Ruby, NuGet, Terraform, or Docker.

A classic but flawed workflow using long-lived tokens.

Workflow with Cloudsmith, GitHub Actions using OICD.

CI/CD platforms like CircleCI and Github actions usually require access to resources on external services, like your artifacts hosted on Cloudsmith’s artifact repository.

Cloudsmith Blog

Secure Docker Image Pulls from Cloudsmith to Kubernetes using OIDC

How to Audit Your Software Supply Chain Security

Improving Observability With Cloudsmith Logs

The Dangers Lurking in Open Source Software

Securely Connect Cloudsmith to your CI/CD using OIDC Authentication

Cloudsmith's Enhanced Security with Policy Management

How to Manage Your Vulnerability Workflows with Cloudsmith

Cloudsmith, the JFrog Bintray Alternative & Replacement