基准修改驱动,使用Erlang VoltDB实现每秒87.7万事务

叫你如何修改 

使用Erlang+VoltDB 每秒87.7万事

Henning Diedrich首次接触VoltDB是了大型网游服器找更好的数据在其 了展MySQL的“悲惨遭遇”后 Henning看上了VoltDB 因 VoltDB比MySQL更适合 游世界的需求最起Henning VoltDB比其它任何数据都适合。当然Henning同在找比Java更适合程序服器的言 如同各位所想Erlang捕了Henning的“芳心” 。 一来Henning来 就万事具只欠Erlang上的VoltDB “ ” 了 。

的建立

Henning的作不可不快 3年后VoltDB的要求 2010年其捐献了 的第一个版本。个 使用起来已逼近完美 唯一欠缺的就是集群主机使用的是同步接。而在2012年VoltDB决定建立一个更大更犀利的版本 已在GitHub上源。

下面将介新的异步 所具的特性 Erlang 完全并行的微程

microprocess 极其的快速、定并适合VoltDB 3 承并整合了先前版本。新 的可靠及一致的高吐量保障建立在多年Erlang MySQL 和Emysql之上 而接池和用列更是采用了最初Electonic Arts 的核心建模其可靠性毋庸置疑。些因素保障了新的Erlang 可以承受峰  并VoltDB服器点分配 。

至于基准脚本的建立 Henning借了之前他VoltDB建立Node.js的 。有了之前的些  Henning很清楚云集群能更好运行需要什 的数据以及必 的地方。

内部构的 基本上完成了期的要求程序的微程将用 中的函数一个用的接器送消息 它将理所有的接工作。求 送以后初始化程将被阻塞在一个同步的接收 当然 它不会阻塞你其它的所有程或者根据需要运行 已有理由 异步模式了果从服器送到 程中的mailbox中 。 本段中所的同步只是  而VoltDB的用使用的仍然是异步模式 只是 的

程在接受中等待

在里你有很多的  比如存在你建立用程去理求送的模式从而保你的初始化程不会被 中其它活打断。那些你不会在意成功与否的写操作在

起后你就可以弃之不理 甚至是阻止接收些操作失后的返回消息。基准用程序

基准基于VoltDB的投票用例 个例子在每个VoltDB都存在原始的用例置中有一个面用于示果每400毫秒修改一次。你可以从VoltDB安装目下examples/voter中找到它

基准需要一些准工作数据需要写入6个手的姓名 然后向服器每个核心送100万的写入事每个事都会 手中随机的一个送去一票。最 示每个参者所得的票数通使用VoltDB的materialized view和ad-hoc 。 在VoltDB

中 ad-hoc属于常 

基准源在 的主目之下的etc/bench中 同你可以在其中 一个 的README.md其中述了基准运行的多个途径。你可以使用以下的代在本地

 当然会很慢

$ git clone git://github.com/VoltDB/voltdb.git voltdb

$ git clone git://github.com/VoltDB/voltdb-client-erlang.git erlvolt

$ cd voltdb/examples/voter&&./run.sh&

$ cd&&cd erlvolt&&make clean all bench

通常会在屏幕上示如下果metal:~ hd$ cd voltdb-3-com/examples/voter&&./run.sh&

[1] 10817metal:~ hd$ Initializing VoltDB. . .

_ __ ____ ____ ____

| | / /___ / / /_/ __ \/ __ )

| | / / __ \/ / __/ / / / __ |

| |/ / /_/ / / /_/ /_/ / /_/ /

|___/\____/_/\__/_____/_____/

--------------------------------

Build: 3.0 voltdb-3.0-95-gfffab2b Community Edition

Connecting to VoltDB cluster as the leader. . .

Initializing initiator ID: 0, SiteID: 0:5

WARN:Running without redundancy (k=0) is not recommended forproduction use.

Server completed initialization.metal:erlvolt hd$ cd&&cd erlvolt&&make clean all benchcleanerlc -W-I . ./include +debug_info -o . ./ebin erlvolt.erlerlc -W-I . ./include +debug_info -o . ./ebin erlvolt_app.erlerlc -W-I . ./include +debug_info -o . ./ebin erlvolt_conn.erlerlc -W-I . ./include +debug_info -o . ./ebin erlvolt_conn_mgr.erl

erlc -W-I . ./include +debug_info -o . ./ebin erlvolt_profiler.erlerlc -W-I . ./include +debug_info -o . ./ebin erlvolt_sup.erlerlc -W-I . ./include +debug_info -o . ./ebin erlvolt_wire.erlerlc -W-I . ./. ./include +debug_info -o . ./. ./ebin bench.erl

Erlvolt Bench 0.9 (client 'VSD')

----------------------------------------------------------------------------------

---------------------------------------------------

Client 'VSD' , voter, 100,000 calls, steady, 200 workers, delay n/a, direct,queue n/a, slots n/a, limit n/a, verbose, "n/a" stats/sec

Hosts: localhost:21212connect . . .preparation . . .

Start at: 2013-02-06 18:56:20. . . . . . . . . . . . . . . . . . . . .

Starting: 2013-02-06 18:56:20calls. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .cool down . . .check writes . . . okresults . . . votes: 100,000 (6 contestants)

. . . . .Edwina Burnam: 16,817

. . . .Jessie Alloway: 16,808

. . .Tabatha Gehling: 16,669

. . . . .Alana Bregman: 16,613

. . . .Jessie Eichman: 16,556

. . . . . .Kelly Clauss: 16,537close pool . . .

Client 'VSD' , voter, 100,000 calls, steady, 200 workers, delay n/a, direct,queue n/a, slots n/a, limit n/a, verbose, "n/a" stats/sec

----------------------------------------------------------------------------------

---------------------------------------------------

Client 'VSD' overall: 14,357 T/sec throughput, 0.00%fails, totaltransactions: 100,000, fails: 0, total time: 6.965sec

Erlvolt 0.3.3, bench started 2013-02-06 18:56:20, ended 2013-02-06

18:56:26, database: +100,000 new votes

[++++++++++++++]

更一步的介 比如在云中行最棒的基  看etc/bench/README.md 或者在 根目下看个面 doc/BENCHMARK-README.html 。

基准 果

当在核心-smb +S 1上运行使用一个12-node VoltDB服器集群Erlang 会跑出每个核心上2.65万TPS 事/秒 。如果将全配的16核心的集群例作客端点每台机器上都会达到26万TPS 。

使用8个点接到1个12点的VoltDB集群每个客端点将达到平均的

10.9689万TPS整个集群将达到87.7517万TPS 。

于个基准 的是 而不是服器 Henning Diedrich并未服器集群做出整。当然随着集群 增大每个客端核心的性能可能会受到EC2的网限制。当然基准只有在服器未 的情况下才会体出效果 而通 “backpressure”也可以最大程度上的防止系 。

境

Henning将一个20点的Amazon EC2 cc2.xlarge拆分成8个Erlang客端和12个VoltDB服器点。M3.2xlarge提供了如下特性

• Cluster Compute Eight Extra Large Instance cc2.8xlarge

• 60.5GiB内存

• 88个EC2算元 2×Intel Xeon E5-2670 8核

• 3370 GB的例存

• I/O性能 10G的以太网

点配置如下

• Ubuntu Server 12.04 LTS for Cluster Instances AMI

• Oracle Java JDK 1.7

• Erlang R15B03

• VoltDB Enterprise Edition 3.0 RC 基准在免的Volt 3.0 Community

Edition版本上也可以完美运行

事相

客端通 用Voter的vote 方法行存 个程肯定不会只行一个写操作取决于你的  可分4-6个操作

• 索用者投票者 的位置 一个 操作

• 用者是否超了他的投票次数一个 操作

• 用者投票的象是否有效一个 操作

• 如果上面全部有效 投票生效一个插入操作

基于以上些每个插入同引起两不同的materialized view的改 的 87.7万的TPS造成了350万的QPS 。

察果和注意点

Henning 里最得注的数字就是CPU核心上的2.65 TPS整合了10万次的操作。可以你数你需要的硬件估量行当算 然要切的是服器的性能一般比EC2例要高的多 。当然里没有用8台主机而是使用EC2例 也是基于EC2例性能的透明性。

英文版

Two regulations promulgated for implementation is in theparty in power for a long time and the rule of law conditions, theimplementation of comprehensive strictly strategic plan,implementation in accordance with the rules and discipline tomanage the party, strengthen inner-party supervision of majorinitiatives. The two regulations supporting each other, the < code> adhere to a positive advocate, focusing on morality is of Partymembers and Party leading cadres can see, enough to get a highstandard; < rule > around the party discipline, disciplinary rulerrequirements, listed as "negative list, focusing on vertical gauge,draw the party organizations and Party members do not touch the"bottom line ".Here, the main from four square face two partyrules of interpretation: the first part introduces two party Revisedregulations the necessity and the revision process; the second partis the interpretation of the two fundamental principles of therevision of laws and regulations in the party; the third partintroduces two party regulations modified the main changes andneeds to grasp several key problems; the fourth part on how tograsp the implementation of the two regulations of the party. <code > and <Regulations > revised the necessity and revisedhistory of the CPC Central Committee the amendment to theChinese Communist Party members and leading cadres honestpolitics several guidelines > and <Chinese Communist Partydiscipline and Punishment Regulations > column 1 by 2015 tostrengthen party laws and regulations focus. Two party regulationsrevision work lasted a Years, pooling the wisdom of the wholeparty, ideological consensus, draw historical experience, respect forthe wisdom of our predecessors,which reflects the unity ofinheritance and innovation; follow the correct direction, grasp thelimited goals, adhere to the party's leadership, to solve the massesof the people reflect a focus on the problem. The new revision ofthe < code > and < rule >, reflects the party's 18 and the eighthsession of the third, the spirit of the fourth plenary session,reflecting the experience of studying and implementing the GeneralSecretary Xi Jinping series of important speech, reflects the party'seighteen years comprehensive strictly practice. (a) revised tworegulations of the party need of < the ICAC guidelines > in < in

1997 Leaders as members of the Communist Party of China cleanpolitics certain criteria (Trial) > based on revised, the promulgationand implementation of January 2010, to strengthen the constructionof the contingent of leading cadres play an important role. Butwith the party to manage the party strictly administering thedeepening, has not been able to fully meet the actual needs.Content is too complicated, "eight prohibition, 52 are not allowedto" hard to remember, and also difficult to put into practice; thesecond is concisely positive advocated by the lack of prohibitedprovisions excessive, no autonomy requirements; the third is bannedterms and discipline law, both with the party discipline, disciplinaryregulations repeat and Criminal law and other laws and regulationsrepeat; the fourth is to "clean" the theme is not prominent, not forthe existing problems, and is narrow, only needle of county-levelleading cadres above. < rule > is in 1997 <Chinese CommunistParty disciplinary cases (Trial) > based on revision, in December2003 the promulgation and implementation, to strengthen theconstruction of the party play very important role.Along with thedevelopment of the situation,which many provisions have beenunable to fully meet the comprehensive strictly administering thepractice needs.One is Ji law,more than half of the provisions andcriminal law and other countries laws and regulations Repetition;two is the political discipline regulations is not prominent, notspecific, for violation of the party constitution, damage theauthority of Party Constitution of misconduct lack necessary andserious responsibility to pursue; third is the main discipline for theleading cadres, does not cover all Party members. Based on theabove situation, need to < the criterion of a clean and honestadministration > and < rule > the two is likely to be morerelevant regulations first amendment. By revising, really put theauthority of Party discipline, the seriousness in the party tree andcall up the majority of Party members and cadres of the partyconstitution of party compasses party consciousness. (II) two partyregulations revision process the Central Committee of the

Communist Party of China attaches great importance to tworegulations revision .Xi Jinping, general books recorded in theFifth Plenary Session of the eighth session of the Central

Commission for Discipline Inspection, on the revised regulations <

>made clear instructions.According to the central deployment, theCentral Commission for Discipline Inspection from 2014 under sixmonths begin study two regulations revision. The Standing

Committee of the Central Commission for Discipline Inspection 4review revised.Comrade Wang Qishan 14 times held a specialmeeting to study two regulations revision, amendment clarifies thedirection,major issues of principle, path and target, respectivelyheld a forum will listen to part of the province (area) secretaryof the Party committee, Secretary of the Discipline InspectionCommission, part of the central ministries and state organs

Department

The first party committee is mainly responsible for people,views of experts and scholars and grassroots party organizationsand Party members.Approved by the Central Committee of theCommunist Party of China, on 7 September 2015, the generaloffice of the Central Committee of the Party issued a notice tosolicit the provinces (autonomous regions,municipalities) Party, thecentral ministries and commissions, state ministries and commissionsof the Party (party), the General Political Department of themilitary, every 3 people organization of Party of two regulationsrevision opinion.Central Commission for Discipline Inspection ofextensive solicitation of opinions, careful study, attracting, formeda revised sent reviewers. In October 8 and October 12,CentralCommittee Political Bureau Standing Committee and the PoliticalBureau of the Central Committee After consideration of the tworegulations revised draft.On October 18, the Central Committee ofthe Communist Party of China formally issued two regulations.Can say, two laws amendment concentrated the wisdom of thewhole party, embodies the party. Second, < code > and <

Regulations > revision of the basic principles of two partyregulations revision work and implement the party's eighteen, teneight plenary, the spirit of the Fourth Plenary Session of theEleventh Central Committee and General Secretary Xi Jinpingimportant instructions on the revised < low political criterion >and <Regulations >, highlighting the ruling party characteristics,serious discipline, the discipline quite in front of the law, based onthe current, a long-term, advance as a whole,with Bu Xiudingindependent < rule > and < rule >.Main principle is: first, adhere

to the party constitution to follow. The constitution about disciplineand self-discipline required specific, awaken the party constitutionof party compasses party consciousness,maintaining the authorityof the constitution.General Secretary Xi Jinping pointed out that"no rules, no side round. Party constitution is the fundamental law,the party must follow the general rules. In early 2015 held theeighth session of the Central Commission for Discipline InspectionFifth Plenary Session of the 16th Central Committee,Xi Jinpingagain pointed out that constitution is the party must follow thegeneral rules, but also the general rules." the revision of the <code > and < rule > is Method in adhere to the regulationsestablished for the purpose of combining rule of virtue is toadhere to the party constitution as a fundamental to follow, theconstitution authority set up,wake up the party constitution andparty rules the sense of discipline, the party constitution aboutdiscipline and self-discipline specific requirements. 4 second is toadhere to in accordance with the regulations governing the partyand the party. The Party of rule of virtue "de",mainly refers tothe party's ideals and beliefs, excellent traditional style. Therevised the < code > closely linked to the "self-discipline", insistingon the positive initiative, for all members, highlight the "vital few",emphasized self-discipline, focusing on the morality, and themajority of Party members and the ideological and moralstandards. The revised < > Ji method separately, Ji, Ji Yan toMethod, as a "negative list", emphasizing the heteronomy, focusingon vertical gauge. Is this one high and one low, a positivereaction, the strict party discipline and practice resultstransformation for the integration of the whole party to observemoral and discipline requirements, for the majority of Partymembers and cadres provides benchmarking and ruler. Third, insiston to. In view of the problems existing in the party at the presentstage, the main problems of Party members and cadres in theaspect of self-discipline and abide by the discipline to make clearlydefined, especially the party's eighteen years strict politicaldiscipline and political rules, organization and discipline and toimplement the central eight provisions of the spirit against the fourwinds and other requirements into Disciplinary provisions.Not onepace reachs the designated position, focusing on in line with

reality, pragmatic and effective.After the revision of majorchanges,major changes in the < code > and < rule >modifiedand needs to grasp several key problems (a) < code > < code >adhere to according to regulations governing the party and partywith morals in combination, for at the present stage, the leadershipof the party members and cadres and Party members in existingmain problems of self-discipline, put forward principles,requirements and specifications, showing Communists noble moralpursuit, reflected at all times and in all over the world ethics fromhigh from low 5 common requirements.One is closely linked tothe "self-discipline", removal and no direct relation to theprovisions of . the second is adhere to a positive advocate, "eightprohibition" 52 are not allowed to "about the content of the"negative list moved into synchronization amendment < cases >.Three is for all the party members,will apply object from theleadership of the party members and cadres to expand to all Partymembers, fully embodies the comprehensive strictly required. Thefourth is prominent key minority, seize the leadership of the partymembers and cadres is the key, and put forward higherrequirements than the ordinary Party members. Five is to simplify,and strive to achieve concise, easy to understand, easy toremember. The revised < code > is the ruling Party since the firstinsists on a positive advocate forAll Party members and the self-discipline norms,moral declaration issued to all members of theparty and the National People's solemn commitment. > < criterionof a clean and honest administration consists of 4 parts, 18,morethan 3600 words.After the revision of the < code >, a total ofeight, 281 words, including lead, specification and Party membercadre clean fingered self-discipline norms, etc. Part 3 memberslow-cost clean and self-discipline, the main contents can besummarized as "four must" "eight code".Lead part, reiterated onideal and faith, fundamental purpose, the fine traditions and workstyle, noble sentiments, such as "four must" the principle ofrequirements, strong tone of self-discipline,The higher request for6 and supervised tenet, the foothold in permanent Bao the party'sadvanced nature and purity, to reflect the revised standardsrequirements.Members of self-discipline norms around the partymembers how to correctly treat and deal with the "public and