sectionubuntu11.10

BigdatacompressionprocessingandvericationbasedonHiveforsmartsubstationZhijianQU,GeCHEN(&)AbstractThecapacityandthescaleofsmartsubstationareexpandingconstantly,withthecharacteristicsofinformationdigitizationandautomation,leadingtoaquantitativetrendofdata.
Aimingattheexistingprocess-ingshortagesinthebigdataprocessing,thequeryandanalysisofsmartsubstation,adatacompressionprocessingmethodisproposedforanalyzingsmartsubstationandHive.
ExperimentalresultsshowthatthecompressionratioandquerytimeofRCFilestorageformatarebetterthanthoseofTextFileandSequenceFile.
ThequeryefciencyisimprovedfordatacompressedbyDeate,GzipandLzocompressionformats.
Theresultsverifythecorrectnessofadjacentspeedupdenedastheindexofclusterefciency.
Resultsalsoprovethatthemethodhasasignicanttheo-reticalandpracticalvalueforbigdataprocessingofsmartsubstation.
KeywordsHive,Smartsubstation,Losslesscompression1IntroductionSmartsubstationactsasanimportantfoundationandpillarofstrongsmartgrid,whichhascharacteristicsofinformationdigitization,networkingcommunicationplat-formandinformationsharingstandardization[1,2],andcompletessomefunctionsofsystemmonitoring,control-lingandprotection,etc.
Atrendofhugeamountofdatawithcharacteristicsoflargescale,complextypes,andwideareadistributionproducedbysmartsubstationmakesthetraditionalrelationaldatabasemoreandmoredifculttoadapttotherequirementsoflargescaledataprocessingfrompowerenterprises[3,4].
Presently,bigdatastorageandprocessingaremostlybasedonlargescaleserverswithrelationaldatabasemanagementsystems,whichneedhugeinvestmentandhaveashortageoflowutilizationratioandpoorscalability.
Therefore,thedesignofpowerdatacenterusingtraditionalsystemisfarfromtherequirementsofbigdatastorage,analysisandprocessing.
Thus,howtoprocessandanalyzemassivedataproducedbysmartsubstationeffectivelybecomesagreatchallenge.
Itisurgenttoresearchoneffectivestoragetechnologyforbigdata[5].
DatawarehouseusingHiveisaninfrastructurebuiltontopofHadoopcloudcomputingframework,withgoodscalabilityandfaulttolerance[6,7],whichcanintegratewithlosslesscompressionalgorithms,suchasBZip2,Deate,GzipandLzo.
ItsunderlyingoperationscanbetransformedintoMapReduceparalleltasks[8–10],anditsapplicationinterfaceusesHQLlanguage,whichprovidestheabilityofquickdevelopment.
Hiveisdifferentfromtherelationaldatabase.
Ithasnospecialdataformats,butithasthreekindsofstorageformats,includingTextFile,SequenceFileandRCFile.
Hiveisdesignedtowardsthequeryandanalysisofmassivedata,whichcanbeusedtobuildadatawarehouseforprocessingbigdataofsmartsubstation.
ConsideringthecharacteristicsofbigdataofsmartsubstationandHive,adatacompressionprocessingmethodbasedonHiveisproposedtosolvethementionedprob-lems.
Experimentalresultsshowthatithasasignicanttheoreticalandpracticalvalueforprocessingbigdataofsmartsubstation.
CrossCheckdate:3February2015Received:27June2014/Accepted:25December2014/Publishedonline:8August2015TheAuthor(s)2015.
ThisarticleispublishedwithopenaccessatSpringerlink.
comZ.
QU,G.
CHEN,SchoolofElectrical&ElectronicEngineering,EastChinaJiaotongUniversity,Nanchang330013,China(&)e-mail:chenge880601@163.
com123J.
Mod.
PowerSyst.
CleanEnergy(2015)3(3):440–446DOI10.
1007/s40565-015-0144-92Hiveandstorageformats2.
1ProcessingowofHiveHiveisintroducedrstlyinordertostudythesmartsubstationbasedonHive.
HiveisanopensourcedatawarehouseprojectwithanextensionbasedonHadoopcloudcomputingplatformpublishedbyapachesoftwarefoundation,thusitsupportsawideofdatatypes,variouskindsofstructuredandunstructureddatawithcomplexandheterogeneousstorageformats[11].
Combinedwiththetraditionalstructuredquerysyntax,HiveitselfdenesHivequerylanguage(HQL),throughtheanalysisofHQLsyntaxbythedriver.
HQLtasksaretransformedintoMapReduceparalleltasks,thustheycantakefulladvantageofthehighperformanceandscalabilityofthecloudcomputingandrealizecomplexprocessingforthebigdata.
MapReduceparallelprocessingowofHiveisshowninFig.
1.
Hadoopdistributedlesystem(HDFS)istheleman-agementfoundationofreadingorwritingdatabasedonHive.
TheuniedmanagementofdistributeddataiscarriedoutbyNamenode,Datanodesandclientapplications.
DataprocessingowbasedonHiveisshowninFig.
2.
NamenodeactsasthemanagementmasterofHDFS.
DatanodesareresponsibleforthedatablocksstorageinHDFS,andreportingtheirstatustoNamenodewiththeheartbeatresponseperiodically.
IftheNamenodedoesnotobtainheartbeatsfromaDatanode,itwillmodifythecongurationforDatanodes'directory,anddeterminewhethertheDatanodeappearsfault.
Ifso,itwillnotgetthedataoperationrequest,thentheclientwillreadthesameblocksfromanotherDatanode,andtheclientapplicationsaccesstothedatainastreamingwayintheHDFSsystem.
Hiveprovidestheapplicationswithcommandlineinter-face(CLI),clientinterface(Client)andwebuserinterface(WUI).
Attributessuchastablename,column,andparti-tionofHivearestoredinmetadatadatabase.
ReadingrequestofdataissenttoNamenodebytheclientprocess,andthentheclientreadsthedatainanFSInputstreamingway,accordingtothedistributionofdatablocksstoredindifferentDatanodes.
WritingrequestofdataissenttoNamenodebytheclientprocess,andthentheclientwritesdatainanFSOutputstreamingwaytodifferentDatanodesspeciedbyNamenode.
2.
2CompressionstorageformatsofHive1)TextFileactsasthedefaultstorageformat,whichcanbecombinedwiththedifferentlosslesscompressionalgorithms,aswellasbedetectedanddecompressedautomaticallybyHive.
2)SequenceFileisakindofbinarylewhichtheHadoopprovides,thedatawillbeserializedinlesintheformof\key,value[pairs.
SequenceFileofHiveinheritsfromtheSequenceFiletheHadoopprovides.
SequenceFileformatanditscompressionwaysareshowninFig.
3.
3)RCFileisaspecialcolumnorientedstorageformat,whichskipstheunrelatedcolumnsinqueryprocess.
Infact,itdoesnotreallyskipunwantedcolumnstojumptothetargetcolumns,butscanthestoredmetadataheaderofeachrowgrouptocompletetheabovefunction.
RCFileanditscompressionwayareshowninFig.
4.
ThissectionintroducestheprincipleofdataprocessingandstorageformatsofHive,whichlaysatheoreticalfoundationforthefollowingsections.
Split0MapReducePart0Split1MapReducePart1Split2MapSplitnMapReducePartmCopyCombineHDFSHDFSDatalevelParserlevelUIlevelCompressSortSortCopyCombineCompressHiveFig.
1MapReduceparallelprocessingowofHiveHDFSFSInput/FSOutputstreamNamenodeJobTracker6.
Close2.
Getdata3.
Reading/Writing1.
Open4.
Reading(MapReduce)5.
Writing(MapReduce)(Datanode)(Datanode)(Datanode)(Datanode)StoreddataHeartbeatresponseRack_2Rack_1HadoopclusterClientCLIWUIHiveInterpreterDriverCompilerOptimizerStoreddataStoreddataStoreddataMetadaaFig.
2DataprocessingowbasedonHiveHeaderSyncRecordSynsRecordSyncRecordSyncRecordRecordlengthKeyValueKeylengthRecordlengthKeyCompressedvalueKeylengthWithoutcompressionWithcompressionHeaderSyncBlockSynsBlockSyncBlockSyncBlockNumberofrecordsCompressedvaluesCompressedkeysCompressedvaluelengthsCompressedkeyslengths(a)Recordcompression(b)BlockcompressionFig.
3SequenceFileformatanditscompressionwaysBigdatacompressionprocessingandvericationbasedonHiveforsmartsubstation4411233ApplicationsofsubstationbasedonHiveSmartgridactsasthefuturedevelopmentdirectionofthepowergrid,whichincludespowergeneration,trans-mission,distribution,conversionanddispatching,etc.
Undoubtedly,smartsubstationisoneofthemostimportantlinksinthepowergird[12–14],whichismainlycomposedofprimaryintelligentelectronicdevice(IED)andsec-ondarynetworkingequipments.
Monitoringandcontrolsystemsplayanimportantroleincompletingtheordinaryoperationofsmartsubstation.
SomemainmonitoringdataofsmartsubstationislistedinTable1.
Inordertodealwiththebigdataproblemsofsmartsubstation,theapplicationsofHivecanbeintegratedintothesystemofthesmartsubstation,whichisdividedintothreelayers(processinglayer,baylayerandsubstationlayer).
Theapplicationsofsubstationlayerarebasedonbaylayerandprocessinglayer,includeSCADAmonitor-ingsystemandsomeothermanagementsystems.
ThemonitoringsystemandmanagementsystemareintegratedwithHive,notonlycancompletefunctionsofautomaticmonitoring,automaticcontrol,auxiliarydecisionandinformationsharing,butalsocancompletefunctionsofbigdataminingandmultidimensionaldataanalysis,etc.
ThestructureofsmartsubstationsystembasedonHiveisshowninFig.
5.
ThedataprocessingowofsmartsubstationbasedonHivecanbelogicallydividedintodatasourcelayer,computinglayer,controllayerandapplicationlayer.
SCADA,datamining,auxiliarydecisionandmultidimen-sionaldataanalysisandotherfunctionscanberealizedbyusingHQLinterfaces.
Fourlogicallayersofdatapro-cessingowinthesmartsubstationbasedonHiveareshowninFig.
6.
4AnalysisofresultsFirstly,cloudcomputingclusterisbuiltonHadoopplatformconstructedinUbuntu11.
10system,composedofaNamenode(Master)andthreeDatanodes(Data1,Data2andData3).
HivedatawarehouseinfrastructureisbuiltontopofHadoop.
DistributedcloudcomputingclusterofHiveisshowninFig.
7.
Secondly,loadthemassivesubstationdataintoHivedatawarehouse.
Take15monitoringsimulationvaluesofsubstationasanexample,tostudythedatacompressionandstorage.
Table1MainmonitoringdataofsmartsubstationDeviceMonitoringdataTransformerGasdischarge,minimmoisturecontentCapacitorCapacitivecurrent,dielectricloss,unbalancedthree-phasevoltageGISPartialdischarge,gaspressureSmartswitcherSwitcheractiontimes,theclosingcoilcurrent,voltageBigdatasharingProcessinglayerBaylayerSubstationlayerApplicationsofHiveMonitoringIEDProtectionIEDTransformerBreakerGISSmartswitcherIntelligentdeviceIntelligentdeviceIntelligentdeviceIntelligentdeviceSCADAAuxiliarydecisionsDataminingHiveclientMultidimensionaldataanalysisHiveclientHiveclientHiveclientHiveclientClusterserverFig.
5StructureofsmartsubstationsystembasedonHiveDataminingAuxiliarydecisionsHQLapplicationinterface(Hivequery)TimeseriesdataMonitoringdataOperationdataControllayerApplicationlayerComputinglayerHadoopclusterTaskTrackerMultidimensionaldataanalysisDatasourcelayerDataofsmartsubstationMapReduceHiveHDFS(Datanodes)DataloadedintoHiveSCADAHiveFig.
6FourlogicallayersofdataprocessingowbasedonHive16bytessyncMetadataheaderUaUbUcIaIb201202203204202212222232242203213223233243204214224234244HDFSblocksRowgroup1Rowgroup2Rowgroupn211212213214221222223224231232233234241242243244RowgroupRCFileformat201211221231241ColumncompressionFig.
4RCFileformatanditscompressionway442ZhijianQU,GeCHEN1234.
1ComparisonofquerytimeTherstexperimentiscarriedoutonthreekindsofstorageformatstostudythequeryefciency.
Thirtymil-lionmonitoringdatarecordsarestoredinthreekindsofstorageformats,respectively.
ThequerytimeofoneeldandeighteldsisshowninFig.
8.
AsshowninFig.
8,comparingwiththequerytimeofoneeldandeighteldsinthreekindsofstorageformats,thequerytimeofRCFileisrelativelyless,thequerytimeofTextFileismiddle,whilequerytimeofSequenceFileisrelativelymore.
4.
2LosslesscompressionHivesupportsBzip2,Deate,GzipandLzocompres-siontype.
Inordertoverifythequeryefciencyaftercompression,thesecondexperimentiscarriedoutunderconditionofvemillionmonitoringrecords,testingthreekindsofstorageformats,i.
e.
,TextFile,SequenceFile(compressedinblockway)andRCFilebyusingfourkindsoflosslesscompression(BZip2,Gzip,DeateandLzo)[15–17],respectively.
ThelosslesscompressionratiosprocessedbydifferentkindsofalgorithmsonthreekindsofstorageformatsbasedonHiveareshowninFig.
9.
AsshowninFig.
9,theBZip2compressionratioishigherthanthoseoftheotherthreekindsoflosslesscompressionalgorithms.
InconditionofRCFilestorageformat,thecompressionratioofRCFilereachesabout81.
3%,approximately3.
5%higherthanthoseofTextFileandSequenceFile.
ThelosslesscompressionratiosofDeateandGzipalgorithmsreachabout73.
4%,whiletheLzocompressionratioreachesabout56.
8%.
QuerytimewithandwithoutdatacompressionsonthreekindsofstorageformatsisshowninFig.
10(selectV001fromtable_namewhereNum=Num_max;selectV001,…,V008fromtable_namewhereNum=Num_max).
ExperimentalresultsshowthatquerytimeofBZip2algorithmisrelativelyhigher,andtheefciencyisreducedbydatacompression.
QuerytimeafterDeate,GzipandLzobecomelessthanthatwithoutcompression,whichimprovesthequeryef-ciency,atthesametime,savingthestoragecapacity.
AlthoughtheBZip2compressiondoesnotimprovethequeryefciency,whendatastoredinRCFilestoragefor-mat,thequerytimeofBZip2almostequalstotheef-ciencywithoutcompression.
ItisshowedthattheRCFileimprovesthequeryefciencytosomeextent.
Basedontheaboveexperimentalresults,bigdataofsmartsubstationcanbestoredintoHiveaftercompressionaccordingtoactualdemands.
4.
3EfciencyanalysisofclusterInHiveclustersystemwithpprocessors,iftheparalleldegreeisatisfyip(i=1,2,,n),withoutconsideringtheparalleloverhead,theadjacentspeedupinthesystemcanbedenedsimplyasfollows:MonitoringDatasetNamenodeIntel(R)Core(TM)2AMDAthlon64,AMDAthlonX22.
20GHz,,,2.
0G1.
87GHz,2.
0G2.
53GHz,2.
0GData1Data2Data3HiveclientHDFS(Hadoop)AMDAthlon641.
87GHz,2.
0G172.
16.
11.
11172.
16.
11.
10172.
16.
11.
13172.
16.
11.
12HiveconfigurationsMaster:DatasetDatasetdataOperationdataDatasourceDatanodesFig.
7DistributedcloudcomputingclusterofHive3*107in(onefield)3*107in(eightfields)020406080100120140160180Querytime(s)DifferentfieldsTextFilSequenceFilRCFilFig.
8QuerytimeinthreeformatsOriginaldataLzoGzipDeflateBZip20123456x108DifferentalgorithmsByteswrittenintoHDFSTextFileSequenceFileRCFileFig.
9LosslesscompressionratiosbasedonHiveBigdatacompressionprocessingandvericationbasedonHiveforsmartsubstation443123Sm;npXn=TXnXm=TXmXnTXmXmTXn1whereXmandXnaretheworkloads;T(Xm)andT(Xn)aretheparallelrunningtime.
Consideringtheparalleloverhead,theadjacentspeedupcanbefurtherdescribedas:S0m;npXn=TXnOXnXm=TXmOXmPmj1Xm;j=Vm;jOXmPni1Xn;i=Vn;iOXnXnXmXmPmj1fm;j=fm;jVm;jOXmXnPni1fn;i=fn;iVn;iOXnXnXmEnEm2Fig.
10ComparisonofquerytimeFig.
11Compressionconsumingtimeonthreestorageformats444ZhijianQU,GeCHEN123Forasystemwhichparalleldegreeisi,Xn;ifn;iXn,Xm;jfm;jXm,i1;2;n,j1;2;m;fn;iandfm;jaretheworkloadcoefcients;Vn;iandVm;jarerunningspeed;O(Xn)andO(Xm)areparalleloverheadtime;EnXnXmXmj1fm;jVm;jXnOXm;EmXmXnPni1fn;iVn;iXmOXn:Parallelcomputingshouldbeexecutedasi=pdetimes,thecomputingshouldbegroupedbyptocompletethecomputationofparalleldegreei,wheniislargerthanp,athistimetheadjacentspeedupisdescribedas:S0m;npXmPmj1j=pdefm;jVm;jOXmXnPni1i=pdefn;iVn;iOXnXnXmE0nE0m3wherethevalueofi=pdeistheminimumintegernotlessthani=p.
ParalleloverheadOxwhichisacomplicatedfunctionrelatedwithsoftwareandhardwareandapplicationinclud-inginteractive,communicationalandparalleloverhead.
Infact,manyfactorsimpactontheparallelefciency,therefore,therelativeefciencyincrementcausedbytherelativeamountincrementofdatacanbeusedtoreecttheperformanceoftheclustercomprehensively.
Hence,thefollowingmathematicalformulacanbeobtained:Cm;nplimDx!
0EnEmEm,xnxmxm!
limDx!
0DE=EDx=xlimDp!
0DE=DxE=xxdEdx,ExdlnEdx4wherevariationC(m,n)(p)isacomplexfunctionwhichreectsthecapabilityofrunningprogramsinparallelprocessingsystem,relatedwiththeworkloadX,theserialbottleneck,theloadcoefcientandsomeotherfactors.
OperationsofHQLtasksaretransformedintoMapRe-duceparalleltasks,sothethirdexperimentiscarriedoutinordertotesttheparallelcompressionconsumingtimeinthreekindsofstorageformatsofHive,byusingBZip2,Deate,Gzip,andLzofourkindsoflosslesscompressionalgorithms,respectively.
Takeonemillion,threemillion,vemillion,eightmillion,tenmillion,andtwelvemillionmonitoringdatarecordsasthedataresearchobject,recordtheparallelcompressionconsumingtimeindifferentnumberofdatarecords,thenthecurveofcompressiontimeisdraw,asshowninFig.
11.
ItiscanbeseenfromFig.
11thatthecurvepresentsaconvextrend,thatistosay,thecompressiontimeofthemorerecordsislessthanthatofthelessrecords.
InordertoquantitativelyanalyzethecurveofthecompressiontimeinFig.
11,S0(m,n)(p)andC(m,n)(p)arecalculatedwith(2),(3)and(4).
S0(m,n)(p)andC(m,n)(p)areshowninTable2.
AsshowninTable2,whendatarecordsexceedthreemillion,S0(3,5),S0(5,8),S0(8,10),andS0(10,12)arenotlessthanone,whichmeansthattheprocesseddatasizeinaunitoftimeincreases,compressionefciencyimprovestosomeextent,ascurvesshowninFig.
11that,withdatarecordsincreases,Hadoopclusterhasabettercompressionexecutingefciency,thecompressionefciencyincreasestosomeextent.
C(m,n)(p)showsthatdifferentcompressionalgorithmsondifferentstorageformatscanprovidedetailinformation.
5Conclusions1)StorageformatexperimentsverifythatthequerytimeofRCFileforbigdataisrelativelylessthanthatofTable2S0(m,n)(p)andC(m,n)(p)ofcloudclusterFormatCAS0(1,3)S0(3,5)S0(5,8)S0(8,10)S0(10,12)C(1,3)C(3,5)C(5,8)C(8,10)C(10,12)TFBZip20.
901.
121.
251.
091.
06-0.
060.
180.
420.
320.
3Deate0.
911.
271.
211.
041.
09-0.
050.
410.
350.
140.
48Gzip0.
951.
291.
231.
001.
12-0.
020.
440.
3800.
57Lzo0.
831.
061.
251.
081.
12-0.
090.
090.
420.
280.
65SFBZip20.
851.
181.
231.
091.
15-0.
080.
260.
380.
340.
77Deate0.
931.
291.
141.
091.
06-0.
040.
440.
230.
410.
3Gzip0.
911.
251.
071.
121.
09-0.
050.
380.
140.
480.
52Lzo1.
011.
091.
181.
151.
07-0.
070.
150.
490.
610.
37RCFBZip20.
771.
321.
291.
051.
10-0.
120.
470.
490.
20.
49Deate0.
951.
151.
331.
141.
08-0.
020.
220.
560.
570.
38Gzip0.
861.
181.
351.
091.
09-0.
070.
260.
590.
370.
39Lzo0.
961.
251.
151.
111.
12-0.
020.
380.
250.
460.
64BigdatacompressionprocessingandvericationbasedonHiveforsmartsubstation445123TextFileandSequenceFile,andsobigdataofsmartsubstationcanbestoredwithRCFileformatbecauseofitsbettertimeresponse.
2)LosslesscompressionexperimentsverifythatbigdataofsmartsubstationcanbestoredintoHiveaftercompression,andqueryefciencyofdatacompressedbyLzoishigherthanthatbyGzip,DeateandBZip2,whileBZip2compressionratioofdataisrelativelyhigher.
3)Parallelcompressionexperimentsverifythatwiththedatarecordsincreaseinacertainrange,theclusterhasabetterparallelprocessingefciency,andS0(m,n)(p)andC(m,n)(p)ofcloudclusterfurtherprovethatbigdataprocessingofsmartsubstationbasedonHiveisfeasible.
AcknowledgmentsThisworkissupportedbyNationalNaturalScienceFoundationofChina(No.
51267005)andJiangxiProvinceUniversityVisitingScholarSpecialFundsforYoungTeacherDevelopmentPlan(No.
G201415,No.
GJJ13350).
OpenAccessThisarticleisdistributedunderthetermsoftheCreativeCommonsAttribution4.
0InternationalLicense(http://creativecommons.
org/licenses/by/4.
0/),whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedyougiveappropriatecredittotheoriginalauthor(s)andthesource,providealinktotheCreativeCommonslicense,andindicateifchangesweremade.
References[1]ShaullahGM,OoAMT,ShawkatAliABMetal(2013)Smartgridforasustainablefuture.
SmartGridRenewEnergy4(1):23–34[2]ChenJL,HuangC,ZengZXetal(2012)Smartgridorientedsmartsubstationcharacteristicsanalysis.
In:Proceedingsofthe2012IEEEconferenceoninnovativesmartgridtechnologies—Asia(ISGTAsia'12),Tianjin,China,21–24May2012,4pp[3]Lu¨HL,WangFY,YanAMetal(2012)Designofclouddatawarehouseanditsapplicationinsmartgrid.
In:Proceedingsoftheinternationalconferenceonautomaticcontrolandarticialintelligence(ACAI'12),Xiamen,China,3–5Mar2012,pp849–852[4]ThusooA,SarmaJS,JainNetal(2012)Hive—apetabytescaledatawarehouseusingHadoop.
In:ProceedingsoftheIEEE26thinternationalconferenceondataengineering(ICDE'12),LongBeach,CA,USA,1–6Mar2012,pp996–1005[5]ChuangCC,ChiuYS,ChenZHetal(2013)Acompressionalgorithmfoructuantdatainsmartgriddatabasesystems.
In:Proceedingsofthedatacompressionconference(DCC'13),Snowbird,UT,USA,20–22Mar2013,485pp[6]KaurR,GoyalM(2013)Asurveyonthedifferenttextdatacompressiontechniques.
IntJAdvResComputEngTechnol2(2):711–714[7]KimHM,LeeJJ,ShinMCetal(2009)Amulti-functionalplatformforimplementingintelligentandubiquitousfunctionsofsmartsubstationsunderSCADA.
InfSystFront11(5):523–528[8]WbiteT(2010)Hadoop:thedenitiveguide,2ndedn.
O'Reilly,Sebastopol,pp366–405[9]WangDW,XiaoL(2012)StorageandqueryofconditionmonitoringdatainsmartgridbasedonHadoop.
In:Proceedingsofthe4thinternationalconferenceoncomputationalandinformationsciences(ICCIS'12),Chongqing,China,17–19Aug2012,pp377–380[10]PadhyRP(2012)BigdataprocessingwithHadoop-MapReduceincloudsystems.
IntJCloudComputServSci2(1):16–27[11]KoratVG,DeshmukhAP,PamuKS(2012)IntroductiontoHadoopdistributedlesystem.
IntJEngInnovRes1(2):172–178[12]SongY,LiJR(2012)Analysisofthelifecyclecostandintel-ligentinvestmentbenetofsmartsubstation.
In:Proceedingsofthe2012IEEEconferenceoninnovativesmartgridtechnolo-gies—Asia(ISGTAsia'12),Tianjin,China,21–24May2012,5pp[13]SuYC,WangXM(2010)Researchofdataacquisitionmethodonsmartsubstation.
In:Proceedingsofthe2010internationalconferenceonpowersystemtechnology(POWERCON'10),Hangzhou,China,24–28Oct2010,4pp[14]LiHW(2012)Researchontechnologiesofintelligentequip-mentinsmartsubstation.
In:Proceedingsofthe2012IEEEconferenceoninnovativesmartgridtechnologies—Asia(ISGTAsia'12),Tianjin,China,21–24May2012,5pp[15]KaneJ,YangQ(2012)CompressionspeedenhancementstoLZOformulti-coresystems.
In:ProceedingsoftheIEEE24thinternationalsymposiumoncomputerarchitectureandhighperformancecomputing(SBAC-PAD'12),NewYork,NY,USA,24–26Oct2012,pp108–115[16]PatelRA,ZhangY,MakJetal(2012)ParallellosslessdatacompressionontheGPU.
In:ProceedingsoftheInnovativeparallelcomputingconference(InPar'12),SanJose,CA,USA,13–14May2012,9pp[17]YazdanpanahA,HashemiMR(2011)Asimplelosslesspre-processingalgorithmforhardwareimplementationofdeatedatacompression.
In:Proceedingsofthe19thIranianconferenceonelectricalengineering(ICEE'11),Tehran,Iran,17–19May2011,5ppZhijianQUreceivedtheM.
S.
degreeinSchoolofElectrical&ElectronicsEngineering,EastChinaJiaotongUniversityofChina,Nanchangin2004andPh.
DdegreeinSchoolofElectricalEngineering,BeijingJiaotongUniversityofChina,Beijingin2012.
Hisrecentresearchincludessmartgridinformationnetworkandbiglargedatasetsinformationsystem,andintelligentmonitoringsystem.
GeCHENiscurrentlyapostgraduateinSchoolofElectricalEngineering,EastChinaJiaotongUniversity.
Hisresearchinterestsincludeintelligentdispatchingandinformationsystem,HadoopandHive.
446ZhijianQU,GeCHEN123