Getting Started with Grafana Cloud

Most of us have wrestled with self-hosted Grafana setups - spinning up Docker containers, tweaking configurations, and occasionally wondering if the monitoring tool itself needs monitoring. It’s powerful, yes, but what if there’s an easier way? Enter Grafana Cloud: the all-in-one, scalable, and worry-free solution for observability. No more hosting headaches. Just beautiful dashboards and actionable insights.

In this blog, I’ll walk you through a basic setup on how to connect Prometheus to Grafana Cloud and send metrics seamlessly. But first, let’s peel back the curtain and see how the magic works.

What Happens Under the Hood?

Imagine this: you have Prometheus running locally in a Docker container, diligently scraping metrics from a Blackbox Exporter which is also running locally in a Docker container. Prometheus collects this data and stores it, ready to be queried. Now, the star of the show—remote_write. With a few lines of configuration, Prometheus securely streams these metrics to Grafana Cloud.

Grafana Cloud acts as the brain of your monitoring setup, storing and processing the metrics you send. When you log in, those metrics are at your fingertips, ready to be turned into dashboards, alerts, and all the insights you’ve been dreaming of.

Signing Up and Setting Up

First things first, head over to Grafana Cloud's signup page and create your account. If you're new to Grafana Cloud, don’t worry, it’s quick and free to get started. The free tier is great for learning and small setups, so there’s no need to reach for your credit card just yet.

Once you’re in, it’s time to create what Grafana calls a “stack.” Think of a stack as your personal workspace where all your monitoring magic happens. Here’s how to set it up:

1. Name Your Stack: Give it a name that makes sense for your project. Are you monitoring a personal project? Name it something like MyCoolProjectStack. Working on production systems? Maybe something more professional, like Prod-Monitoring.

1. Select Your Services: Grafana Cloud offers three main services—Metrics, Logs, and Traces. For this setup, we’re focusing on metrics, but you can enable other services if needed. Don’t worry—you can always add more services later as your monitoring needs grow.

2. Configure Your Preferences: Some additional options might pop up, like retention periods and data limits. For the free tier, defaults are often fine, but feel free to explore if you’re curious.

Once you hit “Create,” Grafana Cloud will spin up your stack. This might take a few moments, but when it’s done, you’ll see a dashboard with all the essential details for connecting your tools. These include:

• Prometheus Endpoint: The URL where Prometheus will send its metrics.

• Instance ID and API Key: These act as your credentials, ensuring only you can send data to your Grafana Cloud instance.

Pro Tip: Treat your API key like a secret recipe. Don’t share it with anyone or check it into version control. If it’s ever compromised, regenerate it immediately.

Now, take a moment to pat yourself on the back because, you’ve officially set up your Grafana Cloud stack, and the keys to your observability kingdom are now in your hands! Next, let’s connect Prometheus and start the metrics flow.

Hit "Launch".

Configuring Prometheus

Now that your Grafana Cloud stack is ready, let’s connect Prometheus to start the metrics flow. Think of Prometheus as your data-collector-in-chief. It scrapes metrics from your targets (like the Blackbox Exporter) and needs to forward this data to Grafana Cloud for visualization. This magic happens using Prometheus' remote_write feature.

If you don’t already have Prometheus installed, don’t worry. You can grab it from the official Prometheus website. Follow these simple steps to get it up and running:

1. Download the Prometheus Binary:

   Visit the website, download the latest release for your operating system, and extract the files.

2. Run Prometheus:

   Navigate to the extracted folder and start Prometheus with the default configuration:

    ./prometheus --config.file=prometheus.yml
   

Alternatively, you can use Docker if you prefer containerized setups (What I did in my case):

docker run -d -p 9090:9090 \
  --name=prometheus \
  -v /path/to/prometheus.yml:/etc/prometheus/prometheus.yml \
  prom/prometheus

Configure Prometheus for Remote Write

This is the important bit. Setting up Prometheus (which is running locally) to send metrics to Grafana Cloud. Open your prometheus.yml configuration file (this is where Prometheus keeps all its settings) and add or modify the remote_write section like so:

remote_write:
  - url: https://prometheus-blocks-endpoint.grafana.net/api/prom/push
    basic_auth:
      username: <instance ID>
      password: <API key>

It should look something like this:

Just to make things more clear -

• url: This is the endpoint Grafana Cloud provides. You can find it in your stack details under the “Prometheus” section.

• username: This is the Instance ID from your Grafana Cloud stack.

• password: Use the API key you generated when setting up your Grafana Cloud stack.

Note: Be extra careful while handling your API key. Store it securely in environment variables or secrets managers, especially in production environments.

After saving the changes to prometheus.yml, restart Prometheus to apply the configuration:

• If you’re running Prometheus as a service:

sudo systemctl restart prometheus

• If you’re running it with Docker (like I am):

docker restart prometheus

Prometheus will now start sending all its scraped metrics to Grafana Cloud using the remote_write endpoint. Now, head to Grafana Cloud and test the Prometheus Data Source. You could see that most of them are already configured for you. Just Click on the "Test" button at the end of the page and you're good to go.

Building Dashboards

Now for the fun part. Visualizing your data. In Grafana, navigate to Dashboards > New Dashboard and add a panel. Use PromQL to fetch metrics like:

rate(http_requests_total[5m])

Pick a visualization style, from sleek graphs to eye-catching gauges. Play around, experiment, and create a dashboard that wows your team. (Or you could do what I did)

Grafana has a thriving community that shares prebuilt dashboards for nearly every use case. Importing one is easy:

1. Browse the Grafana Dashboard Library

   Head over to the Grafana Dashboards Repository and search for a dashboard tailored to your needs. For example, look up "Blackbox Exporter" if you want a detailed dashboard for probing metrics.

2. Copy the Dashboard ID

   Each dashboard in the repository has a unique ID. For example, the Blackbox Exporter dashboard might have an ID like 7587. Copy this ID. You’ll need it in the next step.

3. Import the Dashboard
   In your Grafana Cloud instance:

   • Go to Dashboards and click Import.

   • Paste the dashboard ID and click Load.

   • Assign the Prometheus data source you configured earlier.

The dashboard is now part of your stack, complete with panels, visualizations, and PromQL queries tailored to your use case. You can further tweak and customize it to suit your preferences. Whether you’re creating a dashboard from scratch or importing one from the community, Grafana Cloud offers endless ways to visualize your data. Customize the colors, labels, and thresholds to make it uniquely yours. And don’t forget to share your dashboard with your team. They’ll thank you for it!

Wrapping It All Up

You’ve set up Prometheus with Grafana Cloud, built dashboards, and taken your first steps into modern observability. But this is just the beginning!

What’s Next?

• Set Up Alerts: Don’t just watch metrics—get notified when something goes wrong. Use Grafana Cloud’s alerting to integrate with Slack, email, or PagerDuty.

• Explore Logs and Traces: Extend your setup with Loki for logs and Tempo for tracing to get a full picture of your system.

• Import Community Dashboards: Save time by using pre-built dashboards for tools like NGINX, Kubernetes, and MySQL.

• Try Synthetic Monitoring: Test uptime and performance from different regions with Grafana Cloud’s Synthetic Monitoring feature.

• Master PromQL: Dig deeper into your data with advanced PromQL queries to uncover trends and anomalies.

Grafana Cloud offers endless possibilities to optimize and scale your monitoring. Dive into the documentation and take your setup to the next level. Happy monitoring!