Update
Bloom language now available for download via the "Bud" DSL for Ruby.
Mission
Enabling the construction of
data-rich systems at unprecedented scales using a minimum of software.
About
Many say that "The Cloud" will be the next game-changing
computing platform. The race is on to define and capture it.
Historically, new platforms take off when
independent developers start to make innovative use of
their unique features. In the case of Cloud Computing,
that means exploiting distributed systems in a
datacenter. Up to now there's been no widely-used programming
model that lets developers easily coordinate the
distributed power of a datacenter.
Enter BOOM, an effort to explore implementing
Cloud software using disorderly, data-centric languages. BOOM stands
for the Berkeley Orders Of Magnitude project, because
we seek to enable people to build systems that are OOM
bigger than are building today, with OOM less effort than
traditional programming methodologies.
For more, please visit the FAQ page.
Bloom and Dedalus
Bloom is the language from the BOOM project. It is currently
available in an alpha release as a DSL in Ruby called "Bud".
Bloom is designed to avoid the traditional mismatches between
distributed platforms and sequential programming languages.
It features a "disorderly" approach to program state and logic,
which encourages data-centric parallel thinking, while calling
programmer attention to asynchrony. A key aspect of Bloom is
the use of the CALM principle
to build automatic program analysis and visualization tools for
reasoning about coordination and consistency.
|
Dedalus is a temporal logic language that
serves as a clean foundation for Bloom. The key insight in Dedalus is
that distributed programming is about time, not about space, and
programmers should focus their attention on data, invariants, and
changes in time. Issues of spatial layout are set aside where they
belong: as performance details that need to be addressed as part of
tuning, or managed by an optimizer. Dedalus is an evolution of our
earlier Overlog language, which in turn was based in Datalog. Where
Overlog had complicated operational semantics, Dedalus is pure
temporal logic with no need for the programmer to understand the
behavior of the interpreter/compiler.
BOOM Analytics
As a first concrete exercise in disorderly cloud programming, we built
BOOM Analytics: an API-compliant
reimplementation of Hadoop (BOOM-MR) and the Hadoop File
System (BOOM-FS). Both are written in the Overlog
declarative language, with some modest use of Java for
support. Developed in a relatively short nine-month design
cycle, our new implementation performs competitively with
the standard Hadoop codebase, with a compact and
easily-extendible codebase. Within that timeframe we
extended BOOM with new features not yet available in
Hadoop, including:
- high availability master nodes (via an
implementation of MultiPaxos in Overlog)
- scale-out of master nodes to multiple machines (via simple data
partitioning), and
- unique reflection-based monitoring and debugging facilities (via
metaprogramming in Overlog)
One key to our approach is that everything is data, i.e. rows
in tables that can be queried and manipulated. This includes
persistent data (e.g. filesystem metadata), runtime state (e.g. Hadoop
scheduler bookkeeping), summary stats (e.g. for advanced straggler
scheduling), in-flight msgs and system events, even parsed code. When
everything in a system is data, it becomes easy to do things like
parallelize computations on the state, make it fault tolerant, and
express (and enforce) invariants on legal states of the system.
Please click here to download and run BOOM-FS. A reimplementation
of BOOM-FS in Bud is part of the Bud Sandbox
Sponsors
We would like to thank our sponsors:
- Microsoft
- NTT Multimedia Communications Laboratories
- IBM
- Yahoo!
- National Science Foundation, Grants IIS-0713661, CNS-0722077 and IIS-0803690
- Air Force Office of Scientific Research, Grant FA95500810352
News
|
