News.EOS.WiKi Bilingual News & Info Of EOS

利用dfuse检索和处理区块链数据/Searching and Processing Blockchain Data with dfuse

译文/Translated:

我们对话了dfuse的CEO Marc-Antoine Ross,和他探讨了区块链开发者对高效处理大量数据解决方法的需求。

和我们介绍一下dfuse吧?

Dfuse是一个规模化程度高、用来检索和处理区块链数据的开源平台。Dfuse让开发者能够利用快速、流畅的交互界面搭建现代的区块链应用,带来极好的用户体验。

dfuse抓住了机会,支持目前最高性能的区块链软件EOSIO的启动,并为其提供开源工具,从此,dfuse开始了征途。能够参与EOSIO的建设让我们获得关于最新一代区块链技术的能力和局限的第一手资料。

我们很快就意识到,我们是EOSIO软件社区的技术领袖之一,后来,数百位开发者都找到我们,跟我们述说他们在区块链上搭建应用时遇到的挑战。这个经历很快让我们找到那块“丢失的拼图”,让我们能实现让每个开发者都能接触区块链技术:读取和获得区块链数据的强大的API。

我们迅速帮EOSIO创建了一个查询层,第一次为区块链带来了真正的可搜索性。2018年10月,我们为全球EOSIO开发者社区带来了免费的dfuse公测版。

Dfuse软件栈信息图

What does dfuse offer today? 现在dfuse提供什么服务呢?

Dfuse的主要目标是让和区块链数据交互的过程与和标准数据库交互的过程一样简单。这就意味着我们要建立顾客API,提供开发者习惯的性能好、可靠度高、容易上手的服务。我们有很好地大数据工程师、安全专家和顶级Dev Ops工程师,因此我们知道,我们能够用前所未有的方法把这样的API带给区块链社区。

后来,我们就一直在不断加深我们平台的功能。今天,我们的区块链数据栈有4个产品,通过REST,Websocket,GraphQL 和gRPC APIs提供:

  • dfuse Search:实时、历史的、能识别到分叉的搜索引擎,让你能够利用简单但强大的查询语言检索到区块链的整个交易历史,反应时间甚至不用一秒。
  • dfuse State:提供区块链状态的快照,适用于历史上所有区块高度、实时并能够完全识别分叉。
  • dfuse Lifecycle:追踪交易状态改变时交易的生命周期:进行中、区块内、分叉、替代、失败、过期等。
  • dfuse Push Guarantee: 把交易提交到区块链,并通过一个调用获得确定的不可逆转性,也就不再需要撰写复杂代码处理和恢复复杂的失败案例。

我们利用第一原则搭建dfuse,我们一直在问自己,如果我们不受标准JSON-RPC界面交互限制,我们会希望怎么读取区块链数据。我们提供高阶的区块链API,这比区块链节点提供的本地API效果更好、操作要求更低。

为什么你会选择用区块链技术,又为什么特别选定EOSIO呢?

简单来说,我们在其它高吞吐环境中提供过解决方案,这样的背景让我们选择性能最高的区块链平台:EOSIO。

EOSIO每秒能够完成数千个动作,所以很明显它需要一个强大的解决方案来读取这些数据。除了免费交易这个特点之外,很明显数据(而不单单是价值传递)也会成为EOSIO区块链未来一个很重要的部分。

开发者热爱数据,也需要能够操控数据。但是,其他区块链只是在储存价值而已,它们还没有急迫地需要dfuse这样的产品来处理这个等级的细节。

Dfuse现在处于什么阶段?

dfuse现在已经部署在很多基于EOSIO和以太坊技术的区块链上了(包括公网、联盟链和私链)。我们也积极开发和其它区块链协议的合作。

dfuse为数百个公共和私人部署提供了可规模化的基础架构,包括交易所、交易操作、交互式游戏、社交媒体平台、供给链解决方案和高输出的监管环境。不管你是出于兴趣还是出于跨国公司运营,dfuse都可以满足你的需求。

你要克服哪些问题,你又是怎么做的呢?

EOS公链和现在其它平台都不一样。其交易吞吐量大、区块时间非常短,这两个因素意味着历史提供者要储存的区块链数据可能会快速增长。

因为我们规划合理、富有远见,所以我们能够把我们的基础架构不断扩大,满足区块链的需求,让我们的用户应用能够储存足够的数据来服务其用户。我们一开始设计dfuse的时候就要它每秒能够处理几百万次交易,符合Block.one用EOSIO的要求,所以我们就能不断规模化我们的产品,并让我们的产品可靠。

你有怎样的扩展计划呢?

每天,我们都在和更多的企业用户对话,它们都想集成区块链技术,它们的要求都很独特,但是当我们提供解决方案的时候,这些提升慢慢就成为我们的全球服务。反过来,这也创造了更有活力、功能更加丰富的服务,这就带来了正向的反馈环。我们的计划是不断听取想要区块链数据的用户的需求,为他们找到最好的产品。

我们也在不同的区块链平台开发解决方案,我们听取了整个区块链生态的开发者的需求。我们相信,未来开发者一定可以实现多链开发。开发者不需要学习不同区块链交互和读取的细微差异,dfuse将会充当一个中间层,它会把复杂的细节都去掉,但同时从每个平台中找到好处。

你还参加了哪些EOSIO社区项目?社区对dfuse的反馈怎么样呢?

我们最自豪的是,我们dfuse平台依据Apache 2.0执照开源了我们的区块链数据栈Block.one使用了我们的解决方案。开源dfuse的数据栈让开发者可以免费接入现在性能最好、规模化最高的平台,更好地搭建自己的区块链应用。

它帮助加快在更多区块链平台上集成dfuse,同时还有助于开发诸如低延迟同步、专业索引器、大数据分析和区块浏览器等延展,同时还能促进发现新应用和用例。

此外,dfuse还为EOSIO生态做了很多开源项目。

  • dfuse Community Edition–由EOS Nation启动和管理,dfuse Community Edition是免费的dfuse API集。其任务是要让所有想和EOS公共区块链集成的开发者更好地获得数据。
  • eosq –EOSIO网络上最深入、最精确的区块浏览器。它是dfuse的GUI(图形用户界面),它能获得dfuse中所有的独特的信息。
  • eosc –对任何和EOSIO链交互的开发者来说,它都是最灵活、最强大的命令行工具。
  • eosio.forum – 用来解决EOSIO网络用户异步投票和信息发送的智能合约。这个合约要解决EOS公链的公投系统需求。同时,我们还用它完成Reddit风格的论坛,所有的信息都放在链上。
  • EOS BIOS Boot –我们花了无数日日夜夜才完成了第一个开源工具,通过去中心化的方式启动区块链,从而实现EOSIO超级节点候选人。
  • eosio.unregd –有些以太坊用户在EOS公链启动的时候未能成功注册账户、获得EOS代币,这个智能合约包含了所有这些以太坊用户的以太坊地址信息。

和我们聊聊你的团队,说说他们地特别之处吧?

dfuse团队的创始成员

我们的创始团队是一群连续创业者组成的,大家在大数据、网络安全、DevOp和区块链上有丰富的经验。我们一共创立和出售了6家公司,最近一个公司是卖给了因特尔。我们对区块链技术有激情,我们的目标是通过组织全球的去中心化数据,帮助开发者构建高性能的应用。

dfuse CEO, Marc-Antoine Ross

Marc-Antoine Ross成立并运营过几家公司,还为其它公司担任顾问和投资人。他有两项专利,参与了很多公司的收购,并被收购过两次,第二次是2014年的时候被因特尔收购。几年来,他组成的明星团队被他打造成因特尔内部影响非常大的项目。Marc想再构建一个出色的团队,于是在2018年又启动了他最新的项目dfuse。

dfuse 执行董事长, Richard Reiner博士

Richard是一个连续创业者、公司创办者,他现在在区块链、网络安全、云、物联网这些领域很活跃,在许多高增长的公司担任投资人、顾问、董事会成员,在这些领域中他也曾经创办并成功出售了几个公司,他手上还有无数的专利。

dfuse CTO,Alexandre Bourget

Alexandre喜欢解决复杂的数据问题。他12岁写了第一个僵尸网络、拿着古典钢琴学位毕业,后来在软件工程行业硕果累累,其中包括一些著名的开源项目。Alexandre共同创办过两个新兴企业,包括Bitcredits(比特币支付先驱)。后来他帮PasswordBox(后被因特尔收购)创立了他们的数据栈,最后在因特尔安全消费者部门担任主要数据科学家。现在,Alexandre在区块链领域还很活跃,为几个初创公司提供咨询。

dfuse 运营和策略Mathieu Boulianne

Mathieu之前是国家银行财富管理的投资银行副总裁。他熟悉的领域包括并购交易的买卖双方,几个行业的公共和私人股票发行,但主要关注的还是科技和生物科技领域。他关注区块链领域多年,他是为区块链完全改变金融结构让其更好的能力而被区块链吸引的。

你们还参与了哪些项目呢?

我们已经和几百个活跃的区块链项目合作、给它们支持过,我们有幸合作过的建立在EOSIO上的传奇团队非常多。

我们现在最紧密的合伙伙伴有:

  • Blockchain Service Network (BSN) – BSN由中国国家信息中心领导,它为区块链开发者提供基础架构服务,简化加入过程。开发者能够简单地连接到任何BSN支持的链上,就像今天连接到云服务一样简单。BSN会为开发者提供EOS公链和以太坊主链的dfuse端点,保证它们服务的开发者能够获得行业领先的数据。
  • EOS Nation –EOS Nation是EOS公链地一个超级节点,它有一支强大的技术团队,我们和该团队合作过EOSIO生态中的很多项目。现在我们在合作创建dfuse Community Edition
  • MYKEY –MYKEY是一个跨链和多货币钱包,它让用户能够简单、安全地获取区块链资产。MYKEY总部在中国,这是世界上区块链技术最发达的国家之一,它为想把dfuse集成到自己技术栈的的开发者提供帮助和支持。
  • Rewired.one – Rewired.one为区块链工程师提供企业支持和战略咨询服务,它帮助企业采用和集成EOSIO技术,实现自己的愿景。Rewired.one主要在澳大利亚运营,它把我们和一个充满创新和前瞻性的大陆联系到一起。
  • Ultra – Ultra想要完全颠覆游戏行业,不管是最初的开发和融资,还是游戏内市场和二手数字购买,Ultra想重新分配游戏公司从开发者手上抢走的权力。通过本次合作,Ultra选择了dfuse作为自己的API解决方案,来获得Ultra区块链上的历史数据。
  • WAX –WAX打破了区块链集成的神话,不管是针对游戏开发者还是针对终端用户,WAX让获得游戏内资产变得更加简单、快速、且最重要的是,更容易。为了更好的帮助开发者采用WAX,WAX上的dfuse端点让多个玩家只要变化dfuse的端点,就可以把应用上传到WAX上。这样,当开发者决定做多链部署的时候,进入障碍基本就消失了。

未来您有什么打算呢?

我们预见,dfuse是未来十年的Elasticsearch:它是一个开源平台,让开发者能够进行流和操作基于区块链的数据库和数据流。

我们在建立一个解决方案,让开发者可以更好地查询、流语义,这也是现在区块链节点本地提供无法比拟的(因为节点为了别的目的做了优化)。

Dfuse的目标是帮助在不断发展的基于日志的区块链数据库中做大规模的平行操作,同时还要提高性能,在现在大多数雄心勃勃的协议团队想实现的规模化的基础上更进一步。

请你用一句话概括一下dfuse吧?

Dfuse是一个规模化程度高、用来检索和处理区块链数据的开源平台。

. . .

建立在EOSIO之上?

我们的 #建立在EOSIO 系列展示了利用EOSIO科技的一些卓越的项目,它们都能为我们更加安全和联系的世界添砖加瓦。如果您想给我们下一个阶段的项目提建议,请给我们的开发者关系小组发送邮件spotlight@block.one

–Block.one开发者关系小组

. . .

重要通知:所有提供的材料都受此重要通知的约束,您必须自行熟悉此间条款。该通知包含与我们软件、出版物、商标、第三方资源以及前瞻性声明相关的信息、限制和约束内容。通过访问我们的材料,您接受并同意此通知的条款。

原文/Original:

We spoke with Marc-Antoine Ross, CEO of dfuse, about how blockchain developers need methods to efficiently process large amounts of data.

Could you introduce dfuse for us?

dfuse is a massively scalable open-source platform for searching and processing blockchain data. dfuse empowers developers with the capabilities they need to build modern blockchain applications with fast, fluid interfaces that deliver exceptional user experiences.

dfuse began its journey by seizing an opportunity to support the launch of the most performant blockchain software available, EOSIO, and providing open-source tooling. This involvement gave us first-hand knowledge of both the capabilities and the limitations of the latest generations of blockchain technology.

After quickly positioning ourselves as one of the technical leaders within the EOSIO software community, we were sought out by hundreds of developers who shared their challenges and obstacles when building on blockchain. This experience quickly identified the missing piece of the puzzle to put blockchain within reach of every developer: powerful APIs for reading and accessing blockchain data.

We quickly rolled out a query layer for EOSIO, bringing true searchability to blockchain for the first time ever. In October of 2018, we launched the free public beta of dfuse to the global community of EOSIO developers.

An infographic of the dfuse software stack

What does dfuse offer today?

dfuse’s primary goal was to make interacting with blockchain data as simple as interacting with standard databases. This meant building out custom APIs that delivered the performance, reliability, and high availability that developers were used to. Armed with a team of big data engineers, security experts, and top dev ops engineers, we knew that we could deliver this to the blockchain community in a way that had never been seen before.

Since that time we have been working to further deepen our platform capabilities. Today, our blockchain data stack is composed of 4 products, offered over REST, Websocket, GraphQL and gRPC APIs:

  • dfuse Search: Real-time, historical and fork-aware search engine that enables you to search the entire transaction history of the blockchain using a simple but powerful query language, with sub-second response times.
  • dfuse State: Offers snapshot views of the blockchain state, at any block height historically, in real time and is fully fork-aware.
  • dfuse Lifecycle: Track the lifecycle of a transaction as it changes state: pending, included in a block, forked, replaced, failed, expired, etc.
  • dfuse Push Guarantee: Submit transactions to the blockchain and receive guaranteed irreversibility in a single call, eliminating the need to write complex code to handle and recover from intricate failure cases.

We built dfuse from first principles, asking ourselves how we would want to read blockchain data if we weren’t constrained by the standard JSON-RPC interfaces. We provide higher-level blockchain APIs, which do more, with less effort, than the native APIs provided by blockchain nodes.

Why did you decide to use blockchain technology, and specifically EOSIO?

Simply put, our background in delivering solutions in other high-throughput environments led us to the blockchain platform that had the highest throughput: EOSIO.

Capable of thousands of actions per second, it was clear that EOSIO would need a great solution for reading that data. Coupled with fee-less transactions, it was clear that data (and not just value transfers) was going to be a big part of the future of EOSIO blockchains.

Developers love data, and need to be able to manipulate it. Whereas with other blockchains that were simply stores of value, there was yet to be a pressing need for a product that delivers the level of detail that dfuse does.

What stage is dfuse at?

dfuse is currently deployed on many blockchains based on EOSIO and Ethereum technologies (including public networks, consortium chains, and private chains). Other blockchain protocols are under active development. 

dfuse provides the massively scalable infrastructure for hundreds of public and private deployments, including exchanges, trading operations, interactive games, social media platforms, supply chain solutions, and high-throughput regulated environments. From hobbyist to multinational enterprise, dfuse can keep up with your needs.

What are the challenges you have had to overcome, and how did you do it?

The EOS Public Blockchain is unlike any other platform out there today. Its high transaction throughput and subsecond block times mean that the amount of blockchain data which must be stored by history providers can increase quickly.

Thanks to proper planning and foresight, we have developed our infrastructure to scale with the demands of the blockchain, enabling us to keep our users’ applications fed with the data they need to service their users. By initially architecting dfuse to handle the millions of transactions per second that Block.one strives to deliver with EOSIO, we have been able to scale our offering to provide reliability.

What are your plans for scaling?

Every day, we speak to more Enterprise users who are looking to integrate blockchain technology. Their needs are unique, but as we deliver solutions, those improvements trickle down to our global service. In turn, this creates a more robust and feature-rich service, creating a positive feedback loop. Our plan is to continually listen to those who seek out blockchain data, and figure out how to best deliver it to them. 

We are also developing solutions across different blockchain platforms, listening to the needs of developers across the entire blockchain ecosystem. We believe that in the future, developers will be multi-chain enabled. Rather than having to learn the nuances of interacting and reading from each blockchain, dfuse can be a middle layer that obfuscates complexity, while offering the benefits found from each platform.

What EOSIO community initiatives have you engaged in and how has the community responded to dfuse?

Most proudly, as the dfuse Platform, we open sourced our blockchain data stack under the Apache 2.0 license with Block.one embracing our solution. Open sourcing the dfuse data stack empowers developers to build better blockchain applications with free access to the most performant and scalable platform available.

It helps accelerate development efforts to integrate dfuse on additional blockchain platforms, as well as to develop extensions such as low-latency syncs, specialized indexers, big-data analytics, and block explorers, and help drive the discovery of new applications and use cases.

In addition, dfuse has contributed many open-source developments to the EOSIO ecosystem:

  • dfuse Community Edition – Launched and hosted by EOS Nation, dfuse Community Edition is a freely available set of dfuse APIs. The mission is to provide greater access to data for all developers seeking to integrate with the EOS Public Blockchain.
  • eosq – The most in-depth and high-precision block explorer available for an EOSIO network. A GUI for dfuse, it has access to all of the unique information that dfuse exposes.
  • eosc – The most flexible and powerful command line tool for any developer to interact with an EOSIO chain.
  • eosio.forum – A smart contract to handle asynchronous voting and messaging by account holders on an EOSIO network. Adopted to handle the needs of a referendum system for the EOS Public Blockchain. Also utilized to back a Reddit-style threaded forum, with all messages placed on chain.
  • EOS BIOS Boot – We spent countless days and nights onboarding Block Producer candidates to EOSIO through the first open-source tool to launch a blockchain in a decentralized fashion.
  • eosio.unregd – A smart contract containing the information of all Ethereum addresses of users who did not successfully register their accounts to claim their EOS tokens upon launch of the EOS Public Blockchain.

Can you introduce your team and tell us what makes them/it special?

Founding members of the dfuse team

The founding team is made up of serial entrepreneurs, with deep expertise in big data, cybersecurity, dev ops and blockchain. Combined, we have built and sold 6 companies, with the last one being sold to Intel. We have a passion for blockchain technology, and our goal is to help developers build performant applications by organizing the world’s decentralized data.

dfuse CEO, Marc-Antoine Ross

Marc-Antoine Ross has founded and operated several companies, and advised and invested in others. He has authored two patents, participated in many company acquisitions and was acquired twice, the second by Intel in 2014. Marc turned the rockstar team he assembled over the years into a high impact project within Intel. Marc wanted to build a great team again, and launched his latest venture in 2018 with dfuse.

dfuse Executive Chairman, Dr. Richard Reiner

Richard is a serial entrepreneur and company builder, currently active as an investor, advisor, and Board member with a number of high-growth companies at the intersection of blockchain, cybersecurity, cloud, and IoT, the same spaces in which he has built and successfully exited several companies, and in which he holds numerous patents.  

dfuse CTO, Alexandre Bourget

Alexandre loves to tackle difficult data problems. He wrote his first botnet at 12, later graduated in classical piano, and went on to a prolific career in software engineering, with notable open-source contributions. Alexandre co-founded two startups, including Bitcredits (a bitcoin payments processor). He then helped PasswordBox (acquired by Intel) craft their data stack and ended up as a lead Data Scientist in the Intel Security Consumer division. Today, Alexandre is very active in the blockchain space, advising several early stage companies.

dfuse Operations & Strategy Mathieu Boulianne

Mathieu is a former Vice President in investment banking at National Bank Financial. His expertise is in both buy- and sell-side M&A transactions, as well as public and private equity issuances across several industries, with a focus on the technology and biotechnology sectors. He has been following the blockchain space for years, drawn in by its ability to radically transform financial structures for the better.

What other projects have you partnered with?

Having worked with and provided support for hundreds of active blockchain projects there is a long list of incredible teams building on EOSIO with whom we’ve had the pleasure of partnering.

Our deepest partnerships are currently with:

  • Blockchain Service Network (BSN) – Led by the State Information Center of China, BSN provides infrastructure as a service to blockchain developers in an effort to simplify the onboarding process. Developers will be able to connect to any of the BSN-supported chains as simply as connecting to a cloud service today. BSN will offer dfuse endpoints for developers on both the EOS Public Blockchain and Ethereum mainnet, ensuring industry-leading data access to the developers they empower.
  • EOS Nation – A Block Producer on the EOS Public Blockchain, EOS Nation is a strong technical team with whom we have collaborated on multiple projects within the EOSIO ecosystem. We are currently working with them in their efforts to host the dfuse Community Edition.
  • MYKEY – Cross-chain and multi-currency wallet, enabling simple and secure access to blockchain assets. Based in China, MYKEY provides help and support to developers looking to integrate dfuse into their tech stack in one of the most blockchain-advanced countries in the world.
  • Rewired.one – Providing enterprise support and strategic consulting services for blockchain engineers, Rewired.one has helped enterprises to adopt and integrate EOSIO technology to enable their vision. Based mainly in Australia, Rewired.one keeps us connected to a forward-looking continent filled with innovation.
  • Ultra – With a mission to completely revolutionize the gaming industry, from initial development and funding, to in-game marketplaces and secondhand digital purchases, Ultra seeks to redistribute the power that gaming companies have taken from developers. Through this collaboration, Ultra has selected dfuse as their API solution for accessing historical data on the Ultra blockchain.
  • WAX – Demystifying blockchain integration for the game developer as well as the end user, WAX is making ownership of in-game assets simple, quick, and most importantly, easily accessible. In an effort to greatly help developer adoption, dfuse endpoints on WAX have allowed multiple developers to port their applications to WAX simply by changing from one dfuse endpoint to another. This brought the barrier to entry to almost nothing for a developer when deciding on a multi-chain deployment.

Where do you see dfuse in the future?

We foresee dfuse as being the Elasticsearch of the new decade: an open-source platform for streaming and manipulating blockchain-based databases and data flows. 

We are building a solution to empower developers with better queries and streaming semantics than blockchain nodes can ever natively provide (because nodes are optimized for a different role). 

dfuse aims to enable massively parallel operations on ever-growing log-based blockchain databases and to offer performance that scales beyond what is currently being worked on by the most ambitious protocol teams.

In just one sentence, how do you want readers to remember dfuse? 

dfuse is a massively scalable open-source platform for searching and processing blockchain data.

. . .

Building on EOSIO?

Our #BuiltOnEOSIO series showcases some of the amazing projects leveraging EOSIO technology to build a more secure and connected world. If you would like to suggest a project for us to feature please send an email to spotlight@block.one for our Developer Relations team to review.

Block.one Developer Relations team

. . .

Important Note: All material is provided subject to this important notice and you must familiarize yourself with its terms. The notice contains important information, limitations and restrictions relating to our software, publications, trademarks, third-party resources and forward-looking statements. By accessing any of our material, you accept and agree to the terms of the notice.

原文链接/Original URL:

About the author

By user
News.EOS.WiKi Bilingual News & Info Of EOS

Recent Posts