译文/Translated：
区块链数据库会自然地持续增长。随着每一段新区块的确认，顺序传输的总账变得更长。一个活跃的区块链数据库，其规模在数年时间内能轻易超过数兆字节，而它完整的历史记录则会一直保存着。
对诸多区块链应用程序而言，灵活且可扩展地访问区块链历史数据是基本需求，而与此同时，高效地完成这一过程仍旧是区块链平台所面临的众多挑战之一。由于区块链的历史数据过于庞大，要求访问历史数据的应用程序通常为资源密集型，并且需要特殊的工具，从而让开发人员创建高效的解决方案。
在持续优化EOSIO开发人员所使用的工具的进程中，我们于6月底，在EOSIO 1.8版中推出了State History Plugin的稳定版本。这一历史数据的新方案替代了此前的history_plugin和MongoDB，为访问链上数据提供了更高效和可扩展的途径。此外，今天我们为基于RocksDB的新的更强大的区块链历史工具引入alpha支持，该工具将取代最初包含在历史工具（History Tools）中支持LMDB的alpha版本。为PostgreSQL提供支持的历史工具还将继续。
本章内容概述了EOSIO区块链历史工具方案的整体信息以及一张简约的系统构架说明图，用以阐述可供应用程序开发人员使用的工具。
在优化EOSIO区块链历史记录方案的过程中，我们完成了多重突破。今日的历史工具的构建采用C++编写，用以促进高性能搜索，它能够高效且可扩展地从完整的EOSIO区块链历史记录汇总筛选数兆字节的数据。由于现在已可以获得更高效的工具，我们已摒弃老旧且效率低下的历史数据方案。这包括在EOSIO 1.2版时宣布不再使用的history_plugin，已经最近放弃的MongoDB Plugin。
EOSIO上State History Tools的历史记录
最初的history_plugin方案将数据储存在储存器或RAM中，这意味着随着链的增长，RAM的数量也需要增加。使用RAM服务历史记录变成一项昂贵的方案，因此更加可扩展的方式以MongoDB Plugin的方式诞生。
通过将EOSIO区块链原始的二进制数据转变为JSOM格式，将信息储存在硬盘上，MongoDB Plugin从使用RAM转化为使用硬盘，成为一种具有更高成本效益的方案。但是，这种方式也很快暴露出扩展性问题，随着数据库规模增大，它因存储为JSON纯文本而导致性能下降。随着方案的进化，MongoDB也与Nodeos集成紧密，导致出现整体的扩展性问题。
为作今后的参考，EOSIO过去以及未来有意弃用的完整了列表可在Github这里查看。
可扩展的方案：State History Plugin
State History Plugin是支持Nodeos的最新历史记录方案，并且整个EOSIO开发者社区已接受该插件。它作为消防水带，通过应用程序开发人员首选的用于程序数据的历史工具，传输EOSIO数据。外部程序使用State History Plugin便可跟踪链上数据，包括数据表、交易历史记录及区块。反之，可以将这一数据进行储存、传输或进行索引，用以迎合各种需求。该插件也可将数据储存在便携文件中，区块制作者可以与快照一同分发，以减少启动带有完整状态历史记录的新Nodeos实例所需要的时间。
State History Plugin允许开发人员仅请求向消费者提供不可逆转的数据，无论Nodeos在哪个模式下运行。基于这一因素，开发人员不需要在自身的应用程序中实行逻辑，以查看由于操作数据未完成，交易是否需要进行回滚。这为开发人员提供了有关State History Plugin数据有效性的相对信心。例如，它对于操作依赖交易最终性的交易所运营商或其他平台或服务非常有效。
将Nodeos与State History访问剥离
State History Plugin与Nodeos充分融合而并未紧密集成，从而导致不稳定性。它通过WebSocket提供数据，允许多类历史记录工具磨炼感兴趣的特定数据，并相应地填充开发人员所选择的数据库。
使用State History Plugin，开发人员不再需要在Nodeos中编写一段集成紧密的插件，以实现可靠的历史数据访问。它成为最终区块链事件发布的机制。现在，开发人员可以编写仅需依赖于使用State History Plugin中的数据的应用程序和基础结构。开发人员可以完整地控制自身应用程序所需数据的筛选和转换。
State History Plugin克服了MongoDB模型中出现的限制问题，并为开发人员提供了支持PostgreSQL及RocksDB的新型结构。
与State History Plugin集成的History Tools
State History Plugin与历史记录工具联合运作，旨在通过管理EOSIO区块链完整的历史记录，填充开发者指定的数据库。填充数据库后，高性能搜索工具WASM-Query Language（WASM-QL）可帮助以可扩展的方式查询二进制合同数据。WASM-QL传送数据，因此，开发者访问并从数据库中查询所需的数据。
WASM-QL高性能的查询机制允许进行可扩展的完整历史记录查询。WASM-QL拥有两个组成部分。一个组件在JavaScript应用程序内的客户端浏览器中运行，第二个组件在数据库附近的服务器上运行。与数据库紧密连接让开发人员能够使用智能合约开发人员编写的预定义查询更有效地利用和查询数据。反序列化后，存储在二进制文件中的数据更易于应用程序访问，并且可以在客户端WASM和浏览器应用程序之间高效传输。
History Tools中的RocksDB Alpha
除了History Tools中为RocksDB提供的alpha支持，我们正逐渐放弃支持LMDB，LMDB是History Tools初始alpha中引入的工具。LMDB为开发人员提供了一个选项，可以使用PostgreSQL数据库快速实例化数据库，减少维护所需的开销，但它最适合用于测试，不适合生产规模。
RocksDB代表为EOSIO区块链应用开发人员提供的更加稳定的状态历史数据库方案。RocksDB是一个多功能的嵌入式数据库，不仅比数据库服务器更高效，而且管理更简单。
combo-rocksdb、fill-rocksdb和wasm-ql-rocksdb等应用程序提供了与PostgreSQL数据库中存储和检索数据相同的工具功能：
- combo-rocksdb：填充数据库并回应实时数据库的查询
- fill-rocksdb：用于初始填充数据库但无法回应查询; 一旦赶上链，切换至combo-rocksdb
- wasm-ql-rocksdb：查询未填充的数据库
目前，RocksDB在EOSIO区块链上的部分和完整区块链历史记录表现都明显优于LMDB和PostgreSQL数据库。 RocksDB还比PostgreSQL更有效地存储完整的历史记录。
我们将继续研究用于查询状态历史的开源解决方案，并将在历史工具库下发布其他解决方案。随着EOSIO生态系统的不断扩展，我们致力于创建管理EOSIO区块链状态历史的新方法。
保持连接
EOSIO开发人员社区的支持加强了EOSIO生态系统的组件构建。我们期望在使用历史工具并接收输入时能够对其进行迭代和演化。如果您想提供反馈并与我们的团队更紧密地合作，为开发人员改进EOSIO，您可以向我们的开发者关系团队发送电子邮件：developers@block.one。
为了及时了解今后的发布内容，您也可以在EOSIO网站上订阅我们的邮件列表。我们很高兴能够定期为EOSIO开发人员优化软件的可使用性，我们将持续为区块链技术的大规模应用奠定基础。
. . .
重要通知：所有提供的材料都受本重要通知约束，请您务必自行熟悉条款。通知包含与我们软件、出版物、商标、第三方资源及前瞻性声明相关的重要信息、限制和约束。通过访问我们的材料，您接收并同意本通知。
原文/Original：
By nature, blockchain databases are always growing. With the confirmation of each new block, the ledger of sequential transactions becomes ever longer. An active blockchain’s database can easily grow beyond terabytes in size within years and its entire history must always be preserved.
While flexible and scalable access to historical blockchain data is a basic necessity for many blockchain applications, it remains one of the more difficult challenges for blockchain platforms to efficiently deliver. Because a blockchain’s history is so large, applications that require access to historical data are often resource intensive and require special tooling for developers to create performant solutions.
In the ongoing effort to improve tools available to EOSIO developers, a stable version of the State History Plugin was released as part of EOSIO Version 1.8 at the end of June. This new solution for history replaced the prior history_plugin and MongoDB Plugin enabling more efficient and scalable access to on-chain data. Additionally, today we are introducing alpha support for a new and more robust blockchain history tool based on RocksDB that will replace the alpha release supporting LMDB that was initially included in History Tools. Support for PostgreSQL within History Tools will continue.
This article provides an overview of blockchain history solutions on EOSIO and a simplified architecture diagram explaining tools available for application developers.
We’ve had multiple breakthroughs during our ongoing work to optimize history solutions for EOSIO blockchains. Today’s history tools are built to facilitate highly performant searches, written in C++, that can efficiently and scalably sift through terabytes of data from the full history of EOSIO blockchains. As more efficient tools have become available, we’ve deprecated older and now less effective history solutions. This includes the history_plugin that was deprecated with the announcement of EOSIO Version 1.2 and the recently deprecated MongoDB Plugin.
The History of State History Tools on EOSIO
The original history_plugin solution stored data in memory, or RAM, which meant that as the chain grew, the amount of RAM needed also grew. Using RAM to service history became an expensive solution, so a more scalable approach took the form of the MongoDB Plugin.
By converting raw binary data from the EOSIO blockchain into JSON format and storing this information on disk, the MongoDB Plugin transitioned away from using RAM to using disk, a more cost effective solution. However, this approach also quickly presented scalability issues and as the database grew in size it suffered from decreased performance from being stored as JSON plain text. MongoDB was also very tightly integrated with Nodeos leading to overall stability concerns as the solution evolved.
For future reference, a full list of past and future intended deprecations in EOSIO is available here in Github.
A Scalable Solution: The State History Plugin
The State History Plugin is the latest history solution supported for Nodeos and has been well received across the EOSIO developer community. It acts as a firehose, piping EOSIO blockchain data through an application developer’s preferred history tool for use in their application database. With the State History Plugin, it is possible for any external process to track on-chain data including tables, transaction history, and blocks. This data can, in turn, be stored, transformed, or indexed to suit a variety of needs. The plugin also stores data in portable files, which block producers may distribute alongside snapshots to reduce the time it takes to bring up a new Nodeos instance with a complete state history.
The State History Plugin allows developers to request only data that is irreversible to be provided to the consumer, regardless of the mode Nodeos is operating in. Relying on this factor, developers won’t need to implement logic in their applications to see if transactions need to be rolled back as a result of acting on data that was not finalized. This provides relative confidence to developers as to the validity of data from the State History Plugin. For example, it is useful in the case of exchange operators or any other platform or service where operation hinges on transaction finality.
Decoupling Nodeos from Access to State History
The State History Plugin meshes with Nodeos without being so tightly integrated that it causes instability. It provides the data via a web-socket, allowing various history tools to hone in on particular data of interest, and populate databases of a developer’s choice accordingly.
Using the State History Plugin, developers no longer need to write a tightly integrated plugin into Nodeos to reliably get access to history data. It becomes the mechanism through which finalized blockchain events are published. Now developers can write applications and infrastructure that only need to rely on consuming data from the State History Plugin. Developers can fully control the filtering and transformation of data their applications require.
The State History Plugin overcomes limitations present in the MongoDB model and offers a new architecture to developers that supports PostgreSQL and RocksDB.
History Tools Integrating with the State History Plugin
The State History Plugin works in conjunction with history tools designed to populate developer specified databases by managing full history from the EOSIO blockchain. Once a database is populated, a highly performant search tool, WASM-Query Language (WASM-QL), assists with querying the binary contract data in a scalable way. WASM-QL ships data so that developers can access and query what they need from the database.
WASM-QL’s performant query mechanisms allow for scalable full-history queries. WASM-QL has two components. One component runs in the client browser inside a JavaScript application, the second runs on the server nearby the database. This close proximity to the database makes it possible for developers to more efficiently utilize and query the data using pre-defined queries written by the smart contract developers. Once deserialized, the data stored in binary is more accessible to applications and may be efficiently transmitted between the client WASM and the browser application.
RocksDB Alpha in History Tools
With the addition of alpha support for RocksDB in History Tools, we are moving away from supporting LMDB which was introduced in the initial alpha of History Tools. LMDB presented an option for developers to quickly instantiate a database with less overhead needed for maintenance with a PostgreSQL database, however it was best suited for testing and not ideal for production scale.
RocksDB represents a more robust state history database solution for EOSIO blockchain application developers. RocksDB is a versatile embedded database that is not only more efficient, but also simpler to administer than a database server.
Applications such as combo-rocksdb, fill-rocksdb, and wasm-ql-rocksdb, offer identical functionality of tools used to store and retrieve data from a PostgreSQL database:
- combo-rocksdb: Fills the database and answers queries of a live database
- fill-rocksdb: Used to initially populate the database but cannot answer queries; once caught up with the chain, switch to combo-rocksdb
- wasm-ql-rocksdb: Queries a database that isn’t being filled
Currently, RocksDB outperforms LMDB and PostgreSQL databases by considerable margins for both partial and full blockchain history on EOSIO blockchains. RocksDB also stores the full history much more efficiently than PostgreSQL.
We are continuing to examine open source solutions for querying state history, and additional solutions will be released under the history tools repository. As the EOSIO ecosystem continues to expand, we are dedicated to pioneering new and effective methods of managing state history for EOSIO blockchains.
Stay Connected
Building components of the EOSIO ecosystem is strengthened by support from the EOSIO developer community. We expect to iterate and evolve history tools as they are used and input is received. If you would like to offer feedback and work more closely with our team to improve EOSIO for developers, you can send our developer relations team an email at developers@block.one.
To keep up-to-date with future announcements, you can also subscribe to our mailing list on the EOSIO website. We are excited to be regularly improving the usability of the software for EOSIO developers as we continue to lay a foundation for the mass adoption of blockchain technology.
. . .
Important Note: All material is provided subject to this important notice and you must familiarize yourself with its terms. The notice contains important information, limitations and restrictions relating to our software, publications, trademarks, third-party resources and forward-looking statements. By accessing any of our material, you accept and agree to the terms of the notice.
原文链接/Original URL:
