Observability for continuous vulnerability assessment and remediation
Observability is crucial for cybersecurity strategies, offering a proactive approach to identifying and addressing vulnerabilities.
Observability provides a comprehensive view of an organization's digital environment, enabling continuous assessment and timely remediation of security challenges. [ more ]
Grafana Labs Acquires Assert.ai to Bring AI to Observability - DevOps.com
Grafana Labs acquired Assert.ai for automating configurations and customization of dashboards.
Grafana Labs is previewing AI capabilities for incident management and diagnostics in Grafana Cloud.
Grafana Labs also announced the availability of an Application Observability module and the general availability of Grafana Beyla, an open-source auto-instrumentation project. [ more ]
DSC Weekly 9 May 2023 - The case for AI-human collaboration - DataScienceCentral.com
Announcements
In addition to ever-increasing volumes of data, storage needs have evolved due to increases in remote work, the use of cloud services, and cybersecurity concerns such as ransomware.In Modern Storage Management summit, learn from top industry experts and solution providers about the latest ways to effectively manage the flood of enterprise data.
How New Relic's Kubernetes Agents team decreased release time by 99% with GitHub workflows
New Relic automated their software agent release process for the Kubernetes integration, reducing deployment time from two weeks to one hour per week.
New Relic improved their response to security vulnerabilities by enabling code-scanning tools in their continuous integration pipeline, automatically patching code without human interaction within a week of a fix being available. [ more ]
Enhancing web debugging using source maps with New Relic browser monitoring
Nonfunctional requirements (NFRs) are important indicators of good applications and vital to the success of a system.
Observability is one aspect of achieving nonfunctional requirements (NFRs) and helps with proactive troubleshooting and optimization of software systems. [ more ]
Observability in DevOps is evolving beyond monitoring - Amazic
Observability goes beyond traditional monitoring by understanding system behavior and providing a comprehensive view of system health.
Observability tools help map complex systems, facilitate team collaboration, and integrate automation and AI for proactive performance optimization. [ more ]
Google Previews Log Analytics Feature in Its Cloud Logging Service
Google recently announced the preview of a new feature called Log Analytics in its Cloud Logging service, allowing companies to analyze data collected from their cloud environments.
Chronosphere Acquires Calyptia to Process Log Data More Efficiently - DevOps.com
Chronosphere has acquired Calyptia to enhance its observability platform with additional pipeline management capabilities.
Calyptia's log processors will be embedded into Chronosphere's platform to handle the increasing volume of telemetry data generated by cloud-native applications. [ more ]
Building a Dedicated Platform for Frontend Developers at the Norwegian Government
The Norwegian Labour and Welfare Administration has built a dedicated platform for frontend developers, offering services like a CDN, observability stack, and feature management.
The platform is treated as a product, emphasizing user needs, collaboration, and learning to drive adoption and improve the developer experience. [ more ]
5 Reasons to Move Beyond SRE to Observability - DevOps.com
SRE and observability are often mistakenly equated with monitoring.
The role of an SRE is to define how the engineering team should take ownership of their service and establish a culture focused on infrastructure quality and reliability. [ more ]
Observability is crucial for analyzing the inputs and outputs of LLMs and troubleshooting issues in production.
LLMs like GPT-4 can be limited by the information they were trained on, but frameworks like LangChain and LlamaIndex enable integration with external data sources. [ more ]
Observability goes beyond traditional metrics monitoring and collects various types of data to understand the overall state of the production environment.
Observability helps identify unknown unknowns and can help reduce infrastructure costs. [ more ]
Instrumenting AWS Lambda functions with OpenTelemetry SDKs
OpenTelemetry is an open-source project that provides a unified framework for collecting observability data from distributed systems.
Using OpenTelemetry for serverless applications can help overcome challenges related to ephemeral functions and complex invocation patterns.
OpenTelemetry allows for minimal code changes to enable instrumentation and provides flexibility in configuring sampling rate and span attributes. [ more ]
New Relic AIOps leadership fueled by collaboration with engineers
Within observability, AIOps has emerged as a game changer in the management and operation of complex IT environments, with engineers playing a vital role in safeguarding customer experiences and revenue streams.New Relic is dedicated to providing data to engineers that allows them to fundamentally reshape the way they build and operate critical services so they can optimize their operations, reduce downtime, accelerate innovation, and enhance customer experiences while maintaining engineer happiness.
How to Build Reliability Into Your Data Stack - DZone
On July 26, 2004, a 5-year-old startup by the name of Google was faced with a serious problem: their application was down.For several hours, users across the United States, France, and Great Britain were unable to access the popular search engine.The then-800-person company and its millions of users were left in the dark as engineers struggled to fix the problem and discover the root cause of the issue.
IBM aims to enable multicloud infrastructure control with Hybrid Cloud Mesh | Computer Weekly
Noting that the modern enterprise operates in a hyper-distributed world where users, applications and data can be distributed across multiple locations and environments meaning that they require an offering that improves the network paradigm, IBM has unveiled IBM Hybrid Cloud Mesh, a software-as-a-service (SaaS) offering designed to enable enterprises to bring management to their hybrid multicloud infrastructure.
A herculean effort Of course, both vendors will be working to close these gaps, but they have a long way to go, because this is really hard work!We know firsthand that designing, building, and fine-tuning a highly-scalable, performant, unified observability platform takes years of commitment, focus, and continued investment-and real-world production use-to mature.
Observability: A Computer Weekly Downtime Upload podcast
Tim Thompson-Rye - stock.adobe.cListen to this podcast William Hill's Stephen Wild discusses how the betting website reduces the revenue impact of IT glitches
Stephen Wild, the observability manager at William Hill, runs a 10-strong team that looks after everything going on with the IT at the online bookmaker.
How William Hill's IT copes with big sporting events | Computer Weekly
Online bookies need to cope with the usual peaks and troughs around football, horse racing, motorsports and other major sporting events, but few seem to attract the betting level of the Grand National, and if a site is overloaded or taking too long to respond, punters will take their business elsewhere.
Install the Infrastructure Agent using Amazon EC2 Systems Manager
Given the dynamic capabilities of AWS Systems Manager, management and configuration for Amazon Web Services (AWS) cloud infrastructure is more robust than ever before.Observability is a critical part of infrastructure management-and the AWS Systems Manager makes it easier than ever to install, update, or remove the New Relic infrastructure agent when creating Amazon EC2 instances.
Observability-Driven Development vs Test-Driven Development - DZone
The concept of observability involves understanding a system's internal states through the examination of logs, metrics, and traces.This approach provides a comprehensive system view, allowing for a thorough investigation and analysis.While incorporating observability into a system may seem daunting, the benefits are significant.
New Relic Taps Generative AI to Simplify Observability
By: Mike Vizard on New Relic has added a tool that takes advantage of generative artificial intelligence (AI) to make it simpler to query the telemetry data collected by its observability platform via a natural language interface (NLI).Jemiah Sius, director of developer relations for New Relic, said New Relic Grok will make the New Relic One platform more accessible to a wider range of IT professionals, especially those that haven't mastered New Relic's proprietary programming language that enables DevOps teams to identify the root cause of an issue.
By: Mike Vizard on A global survey of nearly 26,000 DevOps professionals published today by the Continuous Delivery Foundation (CDF) at the cdCon + GitOpsCon event found 84% of respondents are involved in some type of DevOps activity.However, the report also found just under half 49% are using a continuous integration (CI) or continuous delivery (CD) platform and only 22% are using CI and CD approaches to automate all building, testing and deployment of code in a production environment.
New Relic Report Surfaces Spike in Amazon JDK Usage
By: Mike Vizard on An analysis of the Java applications observed by New Relic showed nearly one-third of organizations (31%) are using the Amazon Java development kit (JDK) compared to 28% using the Oracle JDK as more Java applications are built and deployed in the cloud.The report also found the number of Java applications running in containers is remaining steady year-over-year at 70%.
By: Mike Vizard on New Relic is preparing to extend the capabilities of its observability platform by making use of both additional machine learning algorithms and the ChatGPT generative artificial intelligence (AI) platform.Peter Pezaris, senior vice president for strategy and experience at New Relic, said during an interview on an episode of TechStrong TV that a prototype of an AI model created using machine learning algorithms will automatically notify developers when code in a production environment is not performing well-right within their integrated development environment (IDE).
By: Mike Vizard on New Relic today added a free CodeStream module to its observability platform to give developers access to metrics and telemetry data that will enable them to write higher-quality code faster.Peter Pezaris, group vice president and general manager of strategy and experience for New Relic, said this capability is part of an effort to shift observability left toward developers to optimize code as it is written and after it is deployed.
Time to Resolve to Automate More DevOps Workflows in 2023 - Amazic
Heading into 2023 it's already apparent the top priority for most DevOps teams in terms of New Year resolutions needs to be improving productivity.A survey of 500 software and engineering professionals, sponsored by Chronosphere, a provider of an observability platform, found engineers spend, on average, more than 10 hours per week attempting to triage and understand incidents.
Grafana Labs Reins in Cost of Storing Observability Data
By: Mike Vizard on Grafana Labs today made available an ability to customize what observability data is collected via its managed cloud service.This capability can help reduce storage costs in addition to enhancing the dashboard Grafana provides to make it simpler to track metrics usage.Wayne Jin, vice president of product marketing for Grafana Labs, said both capabilities would make it easier for DevOps teams to track and ultimately reduce the cost of storing the time-series data collected via its cloud platform.
A contextual logging solution addresses the challenges of finding a relevant log to your issue and seeing all the logs in one place.Using logs in context works in both directions.You can think of it as circular: You can start with reviewing a specific log and then go to the server.And you can start by looking at data for a server and then go to the right log details.
Application performance monitoring (APM) services are tools that can help you understand how well or poorly your app is performing for your customers, your end users.APM services or products are typically provided by third parties and help you track key information about your application, such as latency, but they also help you understand trends over time and allow your business to achieve specific SLAs (service level agreements) or SLOs (service level objectives).
7 Data Science Trends for 2023, Top ODSC Recordings from 2022, and Python Constants
7 Data Science Trends You Need to Know Going Into 2023 As we begin a new year, here are some of the top data science trends we saw dominate 2022 that we can expect to be prevalent in 2023.Kubeflow MLOps: Automatic Pipeline Deployment with CI/CD/CT Create an advanced Kubeflow pipeline, and automate its deployments and updates with continuous integration, deployment, and training.
Mezmo Adds Free Community Plan for Managing Observability Data
By: Mike Vizard on Mezmo this week added a free trial and a community plan for the Mezmo Telemetry Pipeline service to make it simpler for DevOps teams to store and manage the large amounts of telemetry data they are collecting.Mezmo CEO Tucker Callaway said as more DevOps teams embrace observability to minimize disruptions to application environments, they are struggling to manage the explosion of data being created.
OpenTelemetry metrics is well on its way to approaching general availability, and you'll want to understand this signal to implement it as part of your team's observability strategy.Currently, you can collect some application runtime metrics out of the box with several language SDKs, and you can use the host metrics receiver to generate metrics about your host system.
Observability as Code Part 2: Automate Alerts and Synthetics
As we've discussed in the previous part of this series, observability as code means automating the configuration of your observability tools.You learned the basics of automating dashboards, and now we're going to expand to ways you can apply observability as code approaches to automate both alerts and synthetic monitoring.
Grafana Labs Acquires Pyroscope to Add Code Profiling Capability
By: Mike Vizard on Grafana Labs this week announced it acquired Pyroscope as part of an effort to better integrate Pyroscope's open source continuous profiling project with the open source continuous profiling database that Grafana Labs launched last year.Richard Hartmann, director of community at Grafana Labs, said after logs, metrics and traces, continuous profiling is now the fourth pillar of observability that enables DevOps teams to identify how much infrastructure resources any given piece of code is consuming.
By: Bill Doerrfeld on In today's real-time, hyperconnected reality, data is a core driver behind most digital strategies.User data is constantly gathered and shared across various technology backends and platforms-as is system performance data to inform engineers.Data is pivotal for everything, from generating machine learning models, refining user experiences and even supporting new data-centric services.
Application performance monitoring (APM) and observability are two closely related approaches to understand and improve your software performance, but they don't mean the same thing.APM is one of the steps in a well-rounded observability practice that uses dashboards and alerts for known or expected failures.
Nobl9 Adds Free Tier to SaaS Platform for Managing SLOs
By: Mike Vizard on Nobl9 this week announced it is making available a perpetual free tier of its software-as-a-service (SaaS) platform for managing service level objectives (SLOs).The free tier was announced at the AWS re:Invent 2022 conference and the company also revamped its pricing.Nobl9 Teams Edition (formerly Nobl9 Hydrogen) now includes 50 SLOs, up from 25, for $850/month.
VMware Brings Java 17 Support to Spring Boot Framework
By: Mike Vizard on VMware this week made a Spring Boot 3.0 update for building microservices-based Java applications that is based on the latest long-term support release for the Java Standard Edition (SE) platform.Michael Minella, senior director of software engineering at VMware, said the latest edition of the framework supports Java 17.
InfluxData Delivers Revamped Database for Real-Time Processing
By: Mike Vizard on InfluxData has made good on a promise to deliver a next-generation time-series database that enables observability and monitoring tools used by DevOps teams to ingest high volumes of data more cost-efficiently.In addition, for the first time, the latest iteration of the InfluxDB database supports the de facto standard SQL query language.
By: Mike Vizard on Nobl9 today added a Replay capability to its service level objectives (SLO) management platform that makes it simpler to collect the historical data a DevOps team needs to get started.Nobl9 COO Kit Merker said Nobl9 Replay makes service level indicator (SLI) data instantly available in minutes using 30 days of historical data collected from observability and application performance monitoring (APM) platforms.
JFrog Aligns With New Relic to Improve Observability
By: Mike Vizard on JFrog and New Relic jointly announced today they will provide a free integration tool that will enable New Relic's observability platform to consume log data from JFrog platforms for building and deploying applications securely.Announced at the Futurestack conference hosted by New Relic, this integration eliminates the manual effort that was previously required to integrate these platforms.
The Azure Native New Relic Service is Now Generally Available
Store your telemetry in Azure Your cloud choice matters.Now, with New Relic built on Azure, you can store telemetry data on your preferred cloud.We designed a cell-based platform on Azure that leverages the New Relic service, which is also based on Azure, to meet your need for scalable, cloud-based performance.
By: Alan Shimel on We will be wrapping up our DevOps coverage for the year by the end of this week.It's hard to believe that another year has gone by.The gravitational lensing of COVID-19 makes it hard to see the progression of the last nine years here at DevOps.com.Things seem to get a bit hazy looking back.
Shifting left isn't just about testing your code and automating the build and deployment process.It's about monitoring and testing your CI and CD pipelines, too.In this post, you'll learn how to monitor your GitLab jobs and pipelines using OpenTelemetry and the New Relic GitLab quickstart.After you're finished, you'll be able to:
Track key metrics on your GitLab pipelines in a pre-built dashboard, such as how long your jobs are taking and how often they are failing.
When deploying applications in the cloud, linking your application performance and availability to cloud resources is vital so you can get the right insights quickly.It helps you find and fix issues, discover and implement cloud cost savings, and make data-driven architecture decisions.To aid you in setting up New Relic cloud integrations, we recently added an integration with Terraform.
Cloud Native Microservice To Cloud Agnostic Microservice
https://unsplash.com/photos/BJHN6Do8kjQAs you know, I've been working more with IoT edge use cases around my career.It was going fine with Azure to date, but when we were re-analyzing the architecture, we found a need for cloud-agnostic microservices instead of cloud-native microservices (more precisely with Azure).
Azure Announces Native New Relic Service for Full-Stack Observability
Azure recently announced a native New Relic service for full-stack observability.The performance monitoring service allows monitoring and troubleshooting of cloud applications in real time, providing metrics, traces, and logs.The new service improves observability by consolidating Azure platform logs and Azure Monitor metrics and sending them to New Relic, giving control of resources inside the Azure Portal and helping identify performance issues.
By: Mike Vizard on A survey of 879 enterprise developers and architects from Vaadin, a provider of framework for building Java applications, found that, as part of modernization efforts, the pace at which Java applications are moving to the cloud is now accelerating.The survey found more than one-third (36%) of Java applications are already deployed in the public cloud.
This week in data: The ML, AI and data landscape with Matt Turck
In this week's video, Bruno breaks down the latest edition of the MAD (Machine Learning, Artificial Intelligence and Data) landscape.FirstMark Capital's managing director Matt Turck also joins the CarCast to discuss the current landscape and key trends.Watch the video below for an in-depth explanation of what's happening in machine learning, artificial intelligence and data today: This week's CarCast covers the following:
Financing trends across public and private markets and how they affect the world of data.
Google Releases Kubernetes GitOps Observability Dashboard
Google Cloud introduced Config Management Dashboard that allows the Platform Administrator or Operator to have a real-time picture of configuration syncs and failures across multiple Kubernetes clusters.This will help to simplify the daily critical tasks like reviewing the resources and the progress of configurations to ensure the behaviors are consistent across multiple clusters, and quickly identifies issues and act accordingly.
By proactively monitoring your application, you can resolve issues before they affect your customers.Using New Relic with Pulumi, you can quickly create alerts that you share with your teams, ensuring that you are proactively monitoring your stack.In this post, you'll learn the basics of the following:
What is Pulumi Pulumi versus Terraform
Then you can jump into a hands-on tutorial where you create alerts using Pulumi and New Relic.
GitHub - oslabs-beta/KnightOwl: An npm package of GraphQL middleware to protect you from malicious queries.
KnightOwl is a middleware library to protect your APIs built on express-graphql servers, combined with a web app monitor to give observability to its activity.Sign in at knightowl.app to utilize our web app monitor in conjunction with the library protecting your GraphQL endpoints.(Please make a note of the email and password you use to signup!) Run the command npm i knightowl to install the Knight Owl Rate Limiter, Cost Limiter, and Depth Limiter middleware functions into your express GraphQL codebase.
Module Federation for distributed front ends-the best of both worlds?
Following on from our previous Beamery Engineering article, "Two years of micro-frontends: a retrospective", we want to dig deeper into the fascinating topic of micro-frontends.The right micro-frontends approach can provide many benefits for organisations.However, if poorly architected, this approach can lead to many critical issues, especially:
Overly complex architecture Problems with shared libraries Poor web performance
Beamery's previous article used an example from Facebook for how to build a modern modular React app.
How does Prometheus collect data?A Prometheus server collects metrics data from its targets using a pull model over HTTP.But what exactly is a pull model, and how is it different from a push model?Push versus pull model On a basic level, it really is as simple as it sounds.A Prometheus server pulls telemetry data from somewhere else.
By: Mike Vizard on A survey of 500 engineering and software development professionals in the U.S. suggests more DevOps teams than ever need to resolve to embrace automation at a deeper level in the new year.The survey, sponsored by Chronosphere, a provider of an observability platform, found engineers spend, on average, more than 10 hours per week attempting to triage and understand incidents.
By: Mike Vizard on Opvizor, a unit of Codenotary, has extended its monitoring capabilities to cloud computing environments starting with support for Amazon Web Services (AWS) and Microsoft Azure.Codenotary CTO Dennis Zimmer said Opvizor aggregates data to provide a more efficient approach to applying analytics to surface performance and risk issues.
Kubernetes has been one of the most adopted technologies in the last couple of years.According to the 2021 Cloud Native Survey conducted by " The Cloud Native Computing Foundation " where they received around 3,829 response overall, around 96% of the organizations are using/evaluating Kubernetes.
New Relic Extends Observability Reach to More DevOps Platforms
By: Mike Vizard on New Relic today expanded its ecosystem by adding integrations between its observability platform and a wide variety of DevOps and cybersecurity tools.The New Relic observability platform now integrates with offerings from Atlassian, Amazon Web Services (AWS), CircleCI, Confluent, GitHub, JFrog, Lacework and Snyk and the open source Jenkins continuous integration/continuous delivery (CI/CD) platform.
3. Use stack allocations instead of heap allocations.Java stores data in two different kinds of memory structures: the stack and the heap.Most programmers are more familiar with heaps.A heap is a dynamically-allocated pool of memory that is visible to all threads.It's where the Java Virtual Machine (JVM) typically stores objects and classes.
Using Terraform to generate New Relic dashboards from NRQL queries
The jsondecode() function extracts the data we need from the results and assigns it to the CONFIG variable.The shape of the CONFIG data is slightly different from how it looked in previous parts in order to accommodate the query result data.jsondecode(data.graphql_query.basic_query.query_response).data.actor.account.nrql.results
Logz.io Unveils Managed Open 360 Observability Platform
By: Mike Vizard on At the AWS re:Invent 2022 conference, Logz.io launched an Open 360 platform that combines multiple open source technologies to provide observability across both existing monolithic and emerging cloud-native application environments.Existing Logz.io offerings included in the Open 360 platform are Logz.io
You cannot do GitOps without observability Git is the single source of truth for the system's intended state, while observability provides a single source of truth for the system's actual state Internal observability allows the GitOps controller to identify configuration drift; external observability allows operations staff and other systems to identify changes in the cluster Cloud native observability is a new skill and task you must add to your DevOps team
GitOps is a new software development paradigm that promises to streamline and fully automate the software deployment process.
Mezmo Adds Observability Pipeline to Analyze DevOps Data
By: Mike Vizard on At the KubeCon + CloudNativeCon North America conference this week, Mezmo launched an Observability Pipeline platform that promises to make it simpler to manage, enrich and correlate machine data.Previously known as LogDNA, Mezmo is expanding the scope of its reach to add the ability to augment and analyze data on top of its core log management platform.
New Relic for Startups Expands with Zendesk and Carta
At New Relic, our goal is to democratize access to observability and make observability a daily practice so all engineering teams can monitor, debug, and optimize their entire stack to deliver better customer experiences.Yet we understand that the difference between success and failure for organizations depends on access to the powerful observability tooling and resources needed to build and scale a vision and to keep costs down.
Comprehensive Kubernetes Telemetry with AWS Observability Accelerator
AWS recently created a new template within the AWS Observability Accelerator project that provides an integrated telemetry solution for Elastic Kubernetes Service (EKS) workloads.