slug: mephisto-1.0 title: Announcing Mephisto 1.0 authors: [core]
tags: [releases]
We are super excited to announce Mephisto 1.0!
What's new?
Live Metrics Dasbhoard
Gain insights into your task runs for better observability and debugging. Built upon the rich Prometheus and Grafana ecosystems. To get started, run mephisto metrics install using the CLI tool before your next task.
Model-in-the-loop
We provide an all new RemoteQueryBlueprint for tasks to enable model-in-the-loop capabilities via the ability to invoke remote procedures from the front-end to the back-end. Check out the new example we added at /examples/remote_procedure/mnist/.
Reduced boilerplate
We've vastly reduced the amount of boilerplate code needed to create a Mephisto run script. In some cases, up to a 30% reduction in lines of code. Our examples have been updated to reflect the new workflow.
Validation and Gold units
Improving on our QA functionality, we've introduced validation and gold units. These are launched as additional units during a run to check for worker quality either at the start or throughout the duration of a task. Both are usable by default on Static and RemoteQuery tasks. You can also add these mixins to any Blueprint you write to get this functionality for free.
New Docs Website
We're revamped our docs on our brand new docs website. We've added more tutorials, Python API references, and design documentation.
UI Package Updates
We've updated our package management to be cleaner and scale towards the future. This will allow us to provide more templated and sharable components as we work with teams to build new tasks.
IO protocol revamp
Mephisto's communication layer has been rebuilt from the ground-up to vastly simplify the client/server interface, improve reliability, reduce the number of messages required, and provide clear error messaging for task developers and workers.
...and more
Asyncio Integration + Refactoring: We've migrated to an asyncio-based design under-the-hood to prevent threading and locking issues, reduce invalid states, and prepare us for adopting multiprocessing in future releases. This means Mephisto 1.0 now requires Python >= 3.7.
Other Performance Improvements: Database indexing, SingletonDB by default, and other minor enhancements should improve run performance and the maximum number of concurrent tasks.
Improved Developer Tooling: For Mephisto developers and people interested in working on Mephisto, we've built out new tooling to ensure that our releases are automated/tested, scripts to facilitate entering dev mode, and documentation specifically for you. We've also refactored parts of our underlying codebase to make it easier to jump into and work with.
To upgrade your existing projects to Mephisto 1.0, see our upgrade guide.