#profiling

[ follow ]
#data-quality
fromMedium
4 weeks ago
Software development

Data Quality on Spark, Part 4: Deequ

Deequ is an open-source Spark library by Amazon for expressing, evaluating, and profiling data quality checks at scale, with analyzers and suggestions.
fromMedium
4 weeks ago
Data science

Data Quality on Spark, Part 4: Deequ

Deequ provides scalable, Spark-native tools for defining, profiling, and analyzing data quality checks with Scala APIs and an optional Python wrapper (PyDeequ).
Data science
fromMedium
4 weeks ago

Data Quality on Spark, Part 4: Deequ

Deequ is a Spark-based open-source library for expressing, evaluating, and profiling data quality checks at scale, with analyzers, automatic suggestions, and Scala/Python support.
#python
Philosophy
fromApaonline
2 months ago

Ever Thought Your iPhone Was Listening to You?

Voice assistants do not continuously record conversations; they use wake-word detection, yet collect metadata to build user profiles used for targeting and inference.
Ruby on Rails
fromrubyflow.com
2 months ago

Sentry Ruby SDK 6.0.0 Released

Sentry Ruby SDK adds full logs for Ruby/Rails, tracing and profiling improvements, and introduces breaking changes including dropped support for older Ruby and Rails versions.
Software development
fromMedium
3 months ago

Measure Carefully-Because You'll Fix What You Measure (Based on true story!)

Measure asynchronous work completion, not just scheduling; callbacks can make execution time appear near-zero and hide real costs.
fromdaniel.feldroy.com
3 months ago

Using pyinstrument to profile Air apps

Air is built on FastAPI, so we could use [pyinstrument's instructions](https://pyinstrument.readthedocs.io/en/latest/guide.html#profile-a-web-request-in-fastapi) modified. However, because profilers reveal a LOT of internal data, in our example we actively use an environment variable. You will need both `air` and `pyinstrument` to get this working: ```sh # preferred uv add "air[standard]" pyinstrument # old school pip install "air[standard]" pyinstrument ```
Software development
fromInfoQ
3 months ago

Improved Application Insights Code Optimizations Identify .NET Performance Bottlenecks Automatically

Code Optimizations is an AI-based service running on Azure Application Insights that uses telemetry gathered by the Application Insights Profiler for .NET to analyse runtime behaviour, find performance bottlenecks down to individual methods, and provide actionable suggestions. Developers can view aggregated data over time (defaulting to a rolling 24‑hour window, with history up to 30 days) for their production and non-production environments.
Software development
Python
fromRealpython
5 months ago

Episode #257: Comparing Real-World Python Performance Against Big O - The Real Python Podcast

Real-world performance of algorithms often contradicts Big O expectations, with profiling revealing significant discrepancies.
[ Load more ]