Drescherbuddypress

HowDoesCollaborativeReectionUnfoldinOnlineCommunitiesAnAnalysisofTwoDataSets(E-mail:michael.
prilla@tu-clausthal.
de;oliver.
blunk@tu-clausthal.
de)(E-mail:chounta@ut.
ee)MichaelPrilla1,OliverBlunk1*&Irene-AngelicaChounta2*1ClausthalUniversityofTechnology,Clausthal-Zellerfeld,Germany;2TartuUlikool,Tartu,EstoniaAbstract.
Peoplecanlearnalotthrough(collaborative)reectionatwork:Inorganizations,staffdebateexperiencesandduetoissueseveryday,thusreectingtogetherandlearningfromeachother.
Whilethisisdesirable,itisoftenhinderedbydifferencesintimeandspace.
Onlinediscussionsincommunity-likesystemsmayprovideameanstoovercomethisissueandenablestafftoshareexperiencesandlearnfromthem.
Inthispaper,westudytwodifferentdatasetsfromtwosoftwaresystemstosupportonlinecollaborativereectioninordertoanalyzewhatpossiblefactorsinuencetheoccurrenceofaspectsofreectioninonlinediscussions,likeexperiences,suggestionsandlearning.
Ourresultsincludendingsthatpartiallyapproveexistingmodelsof(collaborative)reection,butalsoothersthataddtoorevenchallengethesemodels.
Overall,wefoundthatcollaborativereectionmaytakedifferentpathstowardslearning.
Fromthiswederivethatfacilitationmechanismsneedtotakeupthesepaths,andthatfacilitationmaybesuccessfulindifferentwaysthananticipatedfromexistingliterature.
Wedescribepossiblefacilitationmechanismsanddiscusstheirimplementation.
Keywords:Collaborativereection,Contentanalysis,Conversationanalysis,Designforreection,Reection1.
IntroductionReectionisaprocessoflearningfromexperiencebygoingbacktopastactivities,re-assessingtheminthelightofcurrentknowledgeanddrawingconclusionsforfurtheractivities(Boudetal.
1985).
Reectionhasbeenrecognizedasanimportantskillinmodernworkplacesandasamindsetofmodernworkforce(Cresseyetal.
2006;Prillaetal.
2013;Raelin2002;Schn1983).
Ithasalsobeenemphasizedthatreectionoftentakesplaceinsocialsettings,inwhichpeopleexplicitlyexchangetheirexperiencesandlearnfromthemtogether(Cresseyetal.
2006;Prillaetal.
2013;Scott2010).
Thissocialorcollaborative(Prillaetal.
2013)reectionhasalotofpotential,asitmayleadtosolutionsforproblemsthatoftengobeyondsolutionsfoundbyindividuals(Hoyrup2004;MercerandWegerif1999).
ThediscourseaboutcollaborativereectionalsolinkstocentralconceptsoftheCSCWcommunitysuchassensemaking,commonground,groupdecisionsupportandcollaborativeproblemsolving(cf.
Prillaetal.
2013,seebelow).
DOI10.
1007/s10606-020-09382-0TheAuthor(s)2020ComputerSupportedCooperativeWork(CSCW)(2020)29:–697741Inthispaper,wefocusoncollaborativereectionatwork,whichisdoneinonlinemediaandismostlyasynchronous:Inmanycases,reectionparticipantsaregeographicallydispersedorlackthetimetoregularlycometogetherforreection.
Supportforcollaborativereectionatworkcanhelpworkerstoimproveasanindividualaswell,justlikewhendoinggroupwork(Prillaetal.
2013).
Thisismirroredbyrecentworkonsupportforreectioninsocialsettings(Marcuetal.
2014;Prillaetal.
2015;Slováketal.
2017;TangandChen2015).
Despitethisworkandrecentinterestinresearchondesignforcorrespondingreectionsupport(Baumer2015),littleisknownaboutsupportforcollaborativereectioninpracticeandatwork(Slováketal.
2017).
Literatureprovidesseveralmodelsexplainingstepsandactivitiesin(collaborative)reection(e.
g.
,Moon1999;FleckandFitzpatrick2010;Krogstieetal.
2013).
However,otherworkprovidescausefordoubtsregardingwhethercollaborativereectionprocessescanbedescribedbythethesemodels(Cresseyetal.
2006;deGrootetal.
2013),andcallsreection'messy'(Cresseyetal.
2006,p.
23)ratherthanstructured(asinthemodels).
Whileitseemsreasonabletodoubtthatcollaborativereectionfollowssingularpathsdepictedinmodels,workavailableonthefacilitation(BlunkandPrilla2017a;Daudelin1996;Fessletal.
2017;Hoyrup2004)showthatcertainaspects,phasesandutterancesareimportantforcollaborativereectionandcanbesupported.
Despitethis,thereisnoworkavailablethatattemptstoidentifythesephasesbeyondtheoreticmodels.
Toclosethisgap,inthispaperweaskthequestionshowdoreectiveconversationsunfoldonlineincommunitiesaswellas(basedonansweringtheformerquestion)howcanwesupportcollaborativereection,andweinvestigatethembasedontwodatasets.
Inparticular,weaimtoshedlightonthedesignofsupportforcollabora-tivereectionbyinvestigatingthecourseofreectionandinuencesoncertainaspectsofreection,likeexperiences,suggestions,andlearninginreectivediscussionsinonlinetoolsandderivinginsightsfromthis.
Welookedatsequencesofcontributionstocollaborativereectionandtriedtondsequencesorcombinationsthatfosterreection.
Bythis,ourworkfocusesonunderstandinghowcollaborativereectionemergesbetweenpeo-ple,thatis,howreectivediscussionsunfoldinonlinecommunities.
Forthisweuseanapproachinspiredbycontentanalysis,andweanalyzetwodatasetswithatotalof135(65and70)conversationscreatedby93(48and45)users.
Morespecically,welookintoindividualconversationsandanalyzehowcertaintypesof(reective)contributionsinuencethecourseofreectivediscussions,lookingatdifferentfactorsthatmayinuencereection.
Thisincludessequenceanalysis,correlationanalysisandregressionanalysisaswellassequentialpatternmining.
Ourresultsincludendingsthatsupportexistingliterature,butalsoshowthatcollaborativereectionmaytakeseveralpathsnottobeanticipatedfromtheliterature.
Wealsondrelationsbetweencontributionstoreectivediscussionsthatchallengeoraddtoexistingmodelsforcollaborativereection.
Fromourresults,wePrillaMichaeletal.
698derivesuggestionsforfacilitationsupportofcollaborativereectionanddiscusshowtoimplementthem.
Ourworkaddstothediscourseonthe(IT-based)supportofhumanpracticesprominentinthe(E)CSCWcommunity.
Collaborativereectionisacommonpracticeatwork,fromwhichalotofpeoplelearnandgrow.
Atthesametimeandlikemanyotherpractices,itisunlikelythatcollaborativereectioncanbemodelledinawaythatdescribeshowreectionunfoldsinonlinecommunities.
Lookingatcollaborativereectionassituatedinasensethatitisalways'messy'andcannotbesupportedbyspecicmeansatallisalsodissatisfying.
Todissolvethisdichotomy,ourworkidentiespathsalongwhichcollaborativereectionunfoldsandprovidesaninitialviewintothemultiplicityofthesepaths.
Inthatway,ittiesinwiththepracticelensof(E)CSCWandprovidesanewwaytolookatcollaborativereectionwhencomparedtoexistingliterature.
2.
Relatedwork2.
1.
CollaborativereectionInlinewiththeunderstandingofreectionprovidedintheintroduction,collabora-tivereectioncanbeunderstoodasreviewingpastactivitiestogetheranddrawingconclusionfromthemtogether(Baumer2015;Cresseyetal.
2006;Prillaetal.
2015).
Experienceisoneofthecornerstonesofreection,aslearnersrefertotheirexperienceinordertolearnfromit(Boudetal.
1985;Schn1983).
Literaturethereforedescribestheprocessofcollaborativereectionasmakingexperiencesexplicit(bye.
g.
writing),sharingandcollaborativelymakingsenseofthem(Dyke2006;FleckandFitzpatrick2010;Scott2010).
Thisneedssupportforindividualarticulationsofexperiences,ideasandreectionaswellasforrelatingtothearticulationsofothers(Daudelin1996;deGrootetal.
2013;Prillaetal.
2015).
Availableliteratureprovidesalotofmodelsandcharacterizationsthataimtoexplainreection(e.
g.
,Boudetal.
1985;Dewey1933;Kolb1984;Moon1999).
OneofthemostcommonmodelsisthereectivelearningcyclebyBoud(seeFigure1).
Itdescribesreectionaslearningfromexperiencesandre-evaluatingthemwithcurrentknowledgeinordertolearnforthefuture(Boudetal.
1985).
Returningtoexperi-encesandre-evaluatingthemisdescribedasaniterativeprocessthateventuallyshouldhaveoutcomessuchasnewperspectivesonexperiencesorchangesinbehavior.
Learningisunderstoodinabandwidththatreachesfromderivingnewinsightsonpastexperiencestochangingbehavior.
Whilethismodelexplainsthecognitiveprocessesofreection,itonlyappliestoindividualreectionanddoesnotsupporttheunderstandingofcollaborativereection.
OthermodelsliketheCSRLmodel(Figure2,left)describereectionwithamoreformalapproachanddeneinputsandoutputsofdifferentstages,e.
g.
gatheringdatatoinitiatethereection,orcreatingaframeforthereectionsessiontoconductit(Krogstieetal.
2013).
ThemodelalsoincludesaspectsofcollaborativereectionHowDoesCollaborativeReectionUnfoldinOnline.
.
.
699suchasarticulatingmeaningcreatedinreectionsessionsbutmakesnoassumptionsonhowcollaborativereectiontakesplaceindetail.
Forreectiontobecomeacollaborativeprocess,thereisaneedtoarticulateandshareexperiences(Daudelin1996;Scott2010)aswellassuggestionsonhowissuescanbesolvedorwhatcanbelearnedfromreectionprocesses(Cresseyetal.
2006;Prillaetal.
2013).
Thisprocess,asarticulatedinthePrilla(2015)modelofcollab-orativereection(Figure2,right),worksasaninterplayofindividual(cognitive)andcollaborative(explicit)reectionactivities,inwhichthearticulationofexperiencesandideasiscrucial.
Despitetheamountandvalueofmodelsandcharacterizationsexplaining(collaborative)reection,ithasalsobeenarguedthatcollaborativereectionisa'complex,multifacetedandmessyprocessthatistamedanddomesticatedattheriskofdestroyingwhatitcanoffer'(Cresseyetal.
2006,p.
23).
Thissuggeststhattheprocessofcollaborativereectioncannot(andshouldnot)bestructured(seealsodeGrootetal.
2013onthis).
Infact,muchofthesupportandmanystudiesavailablefromtheliteratureprovideinsightsonsupportingearlyphasesofreectionsuchasFigure1.
.
TheBoudetal.
(1985)modelofreection.
Figure2.
.
TheKrogstieetal.
(2013)CSRLmodel(left)andthePrilla(2015)collaborativereectionmodel(right).
PrillaMichaeletal.
700datacollectionandexchange,butleavetheprocessofcollaborativereectiontoitsparticipantswithverylittlestructureorintervention(e.
g.
,FleckandFitzpatrick2009;Marcuetal.
2014;Scott2010).
2.
2.
CollaborativereectionandCSCWCollaborativereectionanditssupportareimportanttopicsforHCIand,morespecically,CSCW.
Thisisreectedinthepastdiscourseonsupportforcollabora-tivereection(FleckandFitzpatrick2009;Slováketal.
2017),thevalueofcollab-orativereectioninmethods(BjrnandBoulus2011),theadaptationofsocialgroups(Convertinoetal.
2007)andcollaborativereectionasameansto(re-)designsocio-technicalsystems(Prillaetal.
2013;TangandChen2015).
CollaborativereectioniscloselylinkedtoestablishedconceptsofCSCWandrelatedcommunitiessuchassensemaking(Weick1995),grounding(ClarkandBrennan1991),decisionsupport(Grayetal.
2011)andcollaborativeproblemsolving(RoschelleandTeasley1995),whichhasbeendiscussedinPrillaetal.
(2013).
Asanexcerptfromthatdiscussionthatisrelevantfortheworkpresentedhere,wenotethatcollaborativereectiongoesbeyondconceptssuchassensemakingandgrounding:Asdescribedabove,afterestablishingacommonunderstandingofatopic,reectionistargetedtowardslearningfromthisunder-standing.
Inaddition,collaborativereectioncanbeseenasaspecicmeansforcollaborativeproblemsolvingthatreliesontheperceptionofallreectionpartici-pantsandaimstondasolutionthatstemsfromtheirexperiences.
Moreover,conceptsfromCSCWandrelateddisciplinescanbeusedtoanalyzeandsupportcollaborativereection(seealsobelow).
Forexample,articulationwork(SchmidtandBannon1992;Suchman1996)isanimportantconceptforcollabora-tivereection,whichaffordsexplicitstatementsofexperiencesandideas(Prillaetal.
2012).
Complementingarticulation,reciprocity(Robertson2002)iskeytosuccess-fulcollaborativereection,asreectionparticipantsneedtorefertoeachotherandlinktoothers'statementstoarriveatcommonresults(Prillaetal.
2015).
Articulationandreciprocityarekeyconceptsfortheworkpresentedhere,whichlooksathowreectivediscussionsinonlinecommunitiesunfold,thatis,howpeoplearticulatereectivestatementsandhowtheyrelatetoeachother.
2.
3.
CollaborativereectionandlearningInourwork,weunderstandlearningfromorbycollaborativereectionasinformallearning(Eraut2004)thathappensaspartofpractice(Schn1983).
Thisunder-standingdiffersfromreectionaspartofa(formal)educationprocess,inwhichreectionisaprimaryprocessandforwhichtherearewell-designedproceduresforreection.
Incontrasttothat,in(work)practicereectionoftenisasecondaryprocess(Prillaetal.
2015),whichisconductedinarathermessyinsteadofastructuredorsystematicway(Cresseyetal.
2006).
Asaresult,collaborativeHowDoesCollaborativeReectionUnfoldinOnline.
.
.
701reectionatworkisaprocessthatisdonewheneverthereistime,andthereforeneedsopportunitiesforasynchronouslysharingandrelatingtoexperiencesandideas.
Learningfromcollaborativereectionhappensinprocessesinwhichindividualsarticulatetheirexperiencesandideas(JrvinenandPoikela2001),whentheylinktheirknowledgetoexplicatedexperiencesandideas(Daudelin1996),andwhenpeopledrawconclusionsfromexplicatedexperiencesandideastogether(Hoyrup2004).
InthesenseofStahl(Stahl2000),collaborativereectioncanbeunderstoodasaprocessofcollaborativeknowledgebuildingthatconsistsofindividualandcollaborativelearningactivitiesthatcomplementeachother.
Collaborativereectionresemblesproblembasedlearning(Barrows1986;Hmelo-Silver2004),asbothprocessesrelyonexperiencesinlearning(Dewey1933).
Otherthanfocusingsolelyonproblemsandhowtosolvethem,(collaborative)reectionisusuallytriggeredbyaperceiveddiscrepancybetweenwhatwasassumedandwhathappenedsuchas'contradictinginformation,incon-gruentfeelings,interpersonalconictsandotheroccurrencesduringwork,leadingtoastateofdiscomfortthattheindividualorgroupwantstoovercome'(Krogstieetal.
2013,p.
155),whichisalsocalled'breakdown'(Baumer2015)or'puzzling'(Schn1983).
Thismeansthatifasituationhappeningdoesnotmatchone'spersonalexpectationofhowitshouldhappen,ittriggersreectionduringwhichtriestoanalyzehowandwhatthis'breakdown'isandwhattolearnfromit.
Schnlistslawyerswhoreectduringorafteracourtsessiononhowtheirstrategyhasplayedoutasanexample(Schn1983).
Thismeansthatthespaceforreectioniswiderandlessconcrete(comparedtofocusingona'problem'),andthatreectionoftenhastoincludeaphaseofsensemakingonwhatisreectedupon.
Inaddition,otherthanfocusingon'solving'aproblem(inwhichlearningismostlyconsidereda'by-product'(Eraut2004,p.
250),reectionaimsatcreating'newperspectives'anda'changeinbehavior'(Boudetal.
1985,seeabove)aswellasquestioningassump-tionsthatledtoearlierbehavior(ArgyrisandSchn1978).
2.
4.
Supportforcollaborativereection:stateoftheartDespitetheavailabilityofmodelsforreectionaspresentedabove,supportingcollaborativereection–thatis,accordingtothemodelsabove,outcomessuchasnewperspectives,changesinbehaviorandlearningthatcanbeappliedinpractice–isnotfullyexploredyetintermsofwhattosupportandhowtosupportit.
Thisincreasesthecomplexityofdesigningsupportforsuchreection.
Someapproachesincludetheuseofpicturesasmemoryaidsandreectiontriggers(FleckandFitzpatrick2010)andgenerictoolssuchassharedwhiteboards,onwhichgroupsmayexchangetheirexperiencewiththehelpoffacilitators(KimandLee2002).
Despitethevalueoftheseapproaches,theyprovidetriggersorspacesforcollab-orativereection,butleavethereectionprocesstousers.
Forthesupportofsuchprocesses,scholarsofcollaborativelearninghaveemphasizedtheimportanceofunderstandingdiscoursesinlearningandfacilitatingthemproperly,proposingPrillaMichaeletal.
702supportsuchasguidance(Suthers2000)orscaffolds(Pea2004)aswellassmoothtransitionsbetweenthem(Dillenbourgetal.
2009).
Withregardtosupportforcollaborativereectionbyfacilitation,thisisechoedbyworkonsupportingface-to-facereectiongroups.
Amongthiswork,Daudelin(1996)emphasizesaneedforthefacilitationofcollaborativereectiontostructuretheprocessandgetthemostoutofit(seealsoCresseyetal.
2006)aswellassuggestingquestionstobeaskedinface-to-facemeetings.
Askingtherightquestionsinface-to-facereectionhasbeenreportedtohelppeopletoarticulateexperiencestobediscussed(BjrnandBoulus2011)andtorefertoothersinreection(deGrootetal.
2013).
Withregardtosupportingfacilitationofcollaborativereectioninonlinetools,Davis(2000)suggestspromptsasmeanstoprovokereectivediscus-sions.
Suchpromptsmaybeusedtoperiodicallyremindpeopleofthingstoreecton(Isaacsetal.
2013),toquestiontheirownthinking(LinandLehman1999),toincreasequalityandquantityofcontributions(Renneretal.
2016),andtostructurereectiveinteraction(Davis2000).
However,despitethepotentialtheseapproachesmayhaveforreection,thequestionofwhen,howandforwhattopromptusersoftoolsforcollaborativereectionremains(seeThillmannetal.
2009forasimilardiscussion).
Theworkpresentedinthispaper–besidesothergoals–aimsatprovidinganswerstothesequestions.
2.
5.
ThekeytosupportActivitiesandphasesofcollaborativereectionGeneralmodelsofreectionaspresentedabovehelptounderstandwhatreectionis.
However,theyarenotsufcientfortheimplementationofsupportofcollaborativereectionsuchasfacilitationorguidance,astheydonotdescribeindetailwhichfactorsneedtobepresentincollaborativereectionandwhatcommunicativeactivitiesleadtoitssuccess.
While,forexample,theCSRLmodel(seeFigure2,left)byKrogstieetal.
(2013)containsphasesandinputfactorsthathavetobemetinordertobeabletocontinuethecycle,itfallsshortinprovidingdirectionsforthefacilitationofreectiveconversations.
Tocreatesuchsupport,thereisaneedformoreknowledgeabouthowcollaborativereectiongoeson,whichspecicfactorsinuencecollaborativereectionandhowtheowofcollaborativereectioninonlinediscussionslookslike.
Correspondingworkonreectionhasfocusedonphasesandactivitiesinreectionthatmayleadtolearningoutcomes.
Amongthiswork,FleckandFitzpatrick(2010)describesixcoreactivitiesofreection,includingreturningtoexperiences,sharingthoughtsandofferingalterativeinterpretations.
Moonproposesninestagesofreectionincludingtheexpressionofexperiences,theclaricationofissuesintheexperience,reviewingexperiencesandtransformingideasintoactions(Moon1999).
VanWoerkomandCroon(2008),analyzingdiscussionsinface-to-facegroupsettings,identifyactivitiessuchasobjectingtoacceptedideasinagrouporaskingforfeedbackasdecisivefortheoccurrenceofreectivediscourse.
HowDoesCollaborativeReectionUnfoldinOnline.
.
.
703Inanattempttoanalyzeonlinecollaborativereection,deGrootetal.
(2013)analyzedtypesandlevelsofreectionindifferentcommunities,ndingthat'criticalopinionsharing'iscrucialforfruitfulreectiveconversations.
Findingthatsuchsharingdoesnotoccuroftenandthattherearedifferentfactorsthatinuencewhetheroutcomescanbederivedfromreectiveconversations,theystatethat'criticallyreectivedialoguesisnotasingleconcept'(deGrootetal.
2013,p.
17).
This–asindicatedinsection2.
1–indicatesthattheremaybedifferentwaysinwhichreectiveconversationsunfoldincommunitiesasopposedtophasesoractivitiesthatneedtohappenincertainsequences,butstaysonarathervaguelevelofdescribingtheselevelswithoutprovidingfurtherdetails.
Ourpriorworkbuildsontheworkdescribedinthissectionandaimstoprovidethesedetails,lookingataspectsofreectiveconversationsthatco-occurinsuccessfulandlesssuccessfulthreadscomposedfrominitialstatementswithfollow-up(reective)answers(Prillaetal.
2015).
Whilethisshowsthat,forexample,theexchangeofexperiencesoftenco-occurswithsuggestionsbasedonexperiences,itusescompletethreadsasunitsoftheanalysisandthereforedoesnotshedlightonhowcontributionstoadiscussionthreadinuenceeachother,andhowsequencesofcontributionsleadtotheoccurrenceofreectioninathread.
Thereforeandbecauseofthedifferencesintheliteratureonthistopiclaidoutabove,anotheraimoftheworkinthispaperistolookatthisquestionandndouthowreectiveconversationsunfold,andifthisunfoldingisinlinewithcommonmodelsofreection.
3.
Analysisofonlinecollaborativereectioncontent3.
1.
ResearchquestionsFromthestateoftheartasdescribedearlier,wecanseethatdespitetheamountofvaluableresearchandmodelson(collaborative)reection,thereisagapinavailableknowledge,whichisaboutunderstandinghowreectiveconversationsunfoldinonlinecommunities.
Thisgapleadstodifcultiesinimplementingsupportforthefacilitationofcollaborativereectioninonlinetools.
Literatureoffersseveralmodelsdescribingphasesofcollaborativereectionasdescribedabove.
Usingthesemodels,wemayassumethatcollaborativereectiondevelopsalongacertainpathofarticulatingexperiences,sharingthem,discussingthemandcollaborativelydrawingconclusionsfromthisprocess.
However,thereishardlyanyevidencewhetheronlinereectiveconversationsfollowthesemodels.
Rather,thediversityinherenttohumancommunicationandworkbydeGrootetal.
(2013)questionthis.
ASsuch,therstsetofquestionsguidingourworkfocusesonthenatureofonlinecollaborativereection.
RQ1:HowcanthenatureofonlinecollaborativereectionbedescribedRQ2:(How)DoreectiveconversationsunfoldincommunitiesalongtheelementsmentionedincommonmodelsofreectionPrillaMichaeletal.
704Answeringthesequestionsisdirectedtowardstwodifferentgoals.
Firstthesequestionsaimatputtingexisting(phasebased)modelstothetest,askingiftheycanbeusedtocreatefacilitatingsupportforcollaborativereection.
Second,theques-tionsaimatdiscoveringtheowofreectioninpracticetoinformthissupport.
Inasecondstep,ourworkaimstoidentitytheelementsofreectiveconversationsthatleadtooutcomesfromreectionsuchasnewinsightsandlearning(seethegoalsofreectionasdescribedabove).
Inparticular,weareinterestedinwhatleadstoreectionoutcomessuchassuggestionsandlearnings,andwhatmayhinderthis.
Thenextsetofresearchquestionisconcernedwiththis:RQ1:WhicharetheelementsofreectionthatleadtosuccessfuloutcomesofreectionRQ2:WhicharetheelementsthatdiminishreectionintheconversationsAnsweringthesequestionsmayinformsupportforthefacilitationinthatitmayenabledesignerstobuildfeaturessuchasprompts(seesection2.
2)tofostertheoccurrenceofelementsthatleadtosuccessfulreectionandtherebysupportingreectivityinconversations.
Inaddition,itprovidesinsightsonwhichelementstoavoid,thusinuencingthedesignofsuchfeaturesfurther.
3.
2.
TwodatasetsandthecorrespondingsoftwareplatformsDatasetMcontainsfoursmallerdatasets,collectedusingatoolcalled"TalkReectionApp"thatsupportsindividualandcollaborativereectioninwork-places.
Thetoolwasusedinfourcases,whichbelongtothedomainsofcarehomesandpublicadministrations.
Thetoolwasdesignedtosupportcollaborativereectionamongstitsmembers(forthefollowingalsosee(PrillaandRenner2014)).
Inordertoachievethis,italloweduserstowritedowntheirexperiencesandsharethemwithothers(Figure3,right).
Whileprovidinginitialexperiences,usersalsohadtheopportunitytoprovidearstinitialreection(Figure3,(2)).
Otherscouldthencommentonthediscussion(Figure3,(3)).
Usingthesefeatures,userscouldsharetheirexperiences(Figure3,left)anddiscussamongstthemselves.
Additionally,userscouldsharetheirpostwithindividualusersexclusivelyincasetheydidnotwanttosharesomethinge.
g.
withtheirsuperior.
Thetoolwasnotintegratedintoothersoftwareattheworkplaceofthepeoplereectingtogether.
Workshopswereheldtointroducethetoolstotheusers.
Eachcasehadsmallteamsofbetween9and18users,andinthreeofthefourcases,theplatformwasusedalongsideregularface-to-facemeetings.
Usersmostlykneweachother.
Thetoolwasusedoveratimespanof42to80days.
WedescribedthedatasetMindetailinpreviouswork(Prillaetal.
2015;PrillaandRenner2014).
DatasetEwascollectedinaplatformaimedatcollaborativereectionandlearning(Figure4).
TheusersoftheplatforminprojectEwereemployeesinapublicadministrationorganization.
Startingwith18usersintheinitiallaunchworkshop,over200usershadregisteredtotheplatformafteroneyear.
Ofthese,HowDoesCollaborativeReectionUnfoldinOnline.
.
.
705datasetEcontainsutterancesbythe45userswhocontributedcontenttotheplatform.
TheplatformwasintendedtoconnectacommunityofworkersthroughoutasmallEuropeancountryandthereforemostoftheparticipantsdidnothaveface-to-facemeetings.
Additionally,mostusersdidnotknoweachotherbeforejoiningthecommunity.
TheplatformwasbasedonWordPresswithpluginsforforums(bbPress)andsocialproles(buddyPress).
Usersowneduserprolesandcouldcreateanddiscussin'Groups',whichhaddifferentprivacylevels.
Groupswereusedtoorganizetopicsthematically,e.
g.
bygroupingalltopicsconcerningdiscussionsaboutaspecictypeofcustomer.
Userscouldwritetheirowntopics(Figure4,left)withinthegroups.
Userscouldthennormallydiscussthosetopicsandsharetheirinsights(Figure4,right).
Therearesimilaritiesanddifferencesbetweentheusersengagedincreatingthedatasetsthatweanalyzeinthispaper.
Mostimportantly,datasetMwascreatedbysmallsetsofusers,inwhichpeopleusuallyknewandspecicallyaddressedeachother.
DatasetEstemsfromtheinteractioninalargercommunity,inwhichsomeparticipantsdidnotknoweachotherandthe(initial)sharingofexperienceswasmoreofabroadcastthandirectedtocertainparticipants.
Inbothcase,usersusedtherespectivetoolstoshareexperiencesonandtosolvecurrentissues,butintheparticipantsassociatedtodatasetMtheseissuesweremoreconcrete.
Twoofthegroupsweredealingwithadministrativechangesintheirorganizations,whiletheFigure3.
.
TheTalkReectionAppusedduringthecreationofdatasetM.
Figure4.
.
ThecommunitytoolusedduringthecreationofdatasetE.
PrillaMichaeletal.
706othertworeectedoninteractionswithrelativesoftheirpatients.
Usersusedtheplatformstoposequestionsregardingsituationstheyencounterandhowtodealwiththem.
Othercolleaguestriedtohelpwithaddingtheirexperiencesorpossiblesolutions(basedontheirexperiencesorotherknowledge).
Usersusedthetoolsoverwhelminglyforimportantworkrelatedissuesandonlyatinyfractionofthepostscouldbeconsideredofftopic(e.
g.
wheretomeetforafter-hourdrinks).
WhiletheparticipantsassociatedwithdatasetEsharetheworkingdomainwithtwoofthegroupsindatasetM(publicadministration)andtheoveralltopicwiththeothertwogroups(difcultsocialinteractions,e.
g.
withclients),theirtopicwasmuchwiderandthereforelessconcrete.
Moreover,thedatasetMgroupsusedthetoolforashortertimespanthantheusersassociatedtodatasetE.
ThesedifferencesaresummarizedinTable1andTable2willbeusedfortheinterpretationofresultsbelow.
Asthegroupsstemfromdifferentorganizationsandcountries,therearediffer-encesbetweentheirorganizationalcultures.
Inaddition,theusedtoolsaresimilar,butdifferslightly.
Becauseofthesedifferences,wedonotcombinethedatasets.
WedescribedthedomainandfactorsinuencingtheadoptionofthetoolofdatasetEinearlierwork(BlunkandPrilla2017b).
Inouranalysis,weusethreadsasaunitofanalysis.
Inourcasethreadsarediscussionsconsistingofaninitialtopicandoneormoreanswers.
Ofcourse,theauthoroftheinitialtopiccanalsopostreplieswithinthediscussionthreadsonthecommunitysystems.
Bothdatasetsincludeposts(boththeinitialtopicandreplies)astheirmaincontentaswellasmeta-datasuchaswhopostedwhatandwhen.
DiscussionsareplainastherearenohierarchiesandtheydonotcontainnestedTable1.
Differencesbetweentheuserswhocreatedthedatasets.
CriterionDatasetMDatasetESamplesizeFourgroups,9–18userseach45usersactive,>200totalPurposeDifcultinteractionswithpatients(twogroups,organizationalchange(twogroups)DifcultinteractionswithclientsDomainPublicadministration,medicine/carePublicadministrationTimeofusage42–80daysOneyear(max,someparticipantsjoinedduringtheyear)Table2.
Descriptivestatisticsforbothdatasets.
CategoryDataSetMDataSetE#Users4845(whoactivelycontributed)#Threads6532#Replies/Thread2.
435.
2(onlythreadswithreplies)LongestThread615(numberofposts)HowDoesCollaborativeReectionUnfoldinOnline.
.
.
707replies.
Althoughafeaturefornestedreplieswaspresentinthetools,theusersdidnotuseit.
Bothdatasetsweresubjecttodatacleaning.
Inallcases,threadswhichcontainednon-work-relatedcontentwereremovedfromthedataset(e.
g.
agreeingwheretomeetforafter-hourdrinks).
3.
3.
ContentanalysisFortheanalysisofthethreadsinthedatasetsdescribedabove,wereliedoncontentanalysis.
Contentanalysis(DeWeveretal.
2006)providemeanstogainanunder-standingof(reective)learningdiscoursesbyprovidinga'theoreticalbaseandtheoperationaltranslation'(DeWeveretal.
2006,p.
6)ofthem,andbyrelatingconversationelementstopracticesoflearning(Gee2004).
Contentanalysisusestwoormorecoders,whoapplyacodingschemethatdepictscertainelementstobelookedatintheanalysis.
Itisacommonmethodofunderstandinggroupcommuni-cationinCSCW(e.
g.
Newmanetal.
1995;PrinzandZaman2005)andregardedasthepreferredmethodofanalyzingcommunicationandinteractioniftheamountofmaterialpermitsmanualcoding(IntroneandDrescher2013).
Usingcontentanalysis,weaimatunderstandinghowreectiveconversationsunfoldinonlinecommunities,andhowtoolsmaybeusedtofacilitatethisprocess.
Fortheworkpresentedhere,weusedacontentschemecreatedinearlierwork(Prillaetal.
2015),asitwassuccessfullyappliedfortheanalysisofcollaborativereection.
Besides,theauthorsarenotawareofotherschemesforcontentanalysisofcollaborativereection.
ToolsforautomaticcontentanalysissuchasLIWC(TausczikandPennebaker2010)andEmpath(Fastetal.
2016)donotcontainelementsspecicenoughforadetailedanalysisofreection.
Theschemeweusedcomprisesthethreeimportantstepsofsharingexperiences,reectingonthemandderivinginsightsfromitasdescribedabove.
Withoutgoingintodetails,inTable3wedescribethemostrelevantelementsofthisscheme.
PleaserefertoPrillaetal.
(2015)andAppendixAfordetails.
Amajordownsidetocontentanalysisisthatitcanonlycapturewhathasexplicitlybeenstatedorwrittendown.
Frompreviouswork(deGrootetal.
2013;Prillaetal.
2015),weknowthatthismayprovideaproblemfortheanalysisofreectionwithrespecttolearningoutcomes.
Inthepreviousworkmentionedabove,usersreportedlearningandchangeasoutcomesofreectionsupport,buttheselearningoutcomeswerenotexplicitlyfoundinthedocumenteddiscussions.
3.
4.
CodingprocedureTheschemesusedforcodingdatasetMandEdifferedslightly:IndatasetMwedidnotdifferentiatebetweensuggestionsbasedonexperienceorknowledgeasdescribedabove,butregardedsuggestionsbasedonknowledgeasadvice(seeTable3).
WeintroducedthisdifferentiationfortheanalysisofdatasetE,whichwasdonesometimeaftertheanalysisofdatasetM,inordertodifferentiateplainadvice(withoutPrillaMichaeletal.
708Table3.
Elementsofthecodingschemeusedinouranalysis(cf.
Prillaetal.
2015).
CodeDescriptionEmotions(EMO_*)Emotionshelpuserstounderstandexperiencesofothersandtorelatetothem(Boudetal.
1985;FleckandFitzpatrick2010;Moon1999;Tigelaaretal.
2008).
Wedifferentiatebetweenemotionsinanexperience(providedbytheauthorofacontribution)byusingcodeEMO_OWNandemotionsofotherstowardsthisexperiencebyusingcodeEMO_OTH.
Interpretationofownexperiences(INT)Someauthorsprovidetheirinitialinterpretationswhilewritingthecontribution,whichmayhelpotherstomakesenseofthem(Raelin2002).
WeusecodeINTfortheseutterances.
SharingExperiences(EXP)vs.
Shar-ingKnowledge(KNO)Forcollaborativereection,providingexperiencesneedstobeseparatedfromcontributionsstemmingfromknowledge,asonlytheformerconstitutesareectionprocess(Zhu1996).
Therefore,forcontributionsmentioningexperiencescodeEXPisused,andforcontributionsrelyingonknowledgeorrules(notexplicitlybasedonpersonalexperiences)weusedcodeKNO.
(Dis)Agreement(AGR*)Collaborativereectionbenetsfromchallengingorsupportingexperiences,ideasandsuggestions(Raelin2002;Tigelaaretal.
2008;FleckandFitzpatrick2010).
TodifferentiateagreementfromdisagreementweusedcodesAGRandDISAGR.
ProvidingAdvice(ADV)vs.
Sugges-tions(SUG*)Forreectiontobemorethanruminating,itisvitaltodifferentiatesuggestionswithoutexplanationfromsuggestionsbasedonexperiences(HattonandSmith1995).
Thereisalsoaneedtodifferentiatesuggestionsmadebasedonexperiencesfromthosebasedonknowledge.
Inthecodingscheme,advicearticulatedwithoutfurtherexplana-tionismarkedbycodeADV,solutionsbasedonexperiencesbroughtaremarkedbySUG_EXP,andsolutionsrelatedtoknowledgeornotrelatedexplicitlyonexperiencesarecodedwithSUG_KNO.
Learning(*_LOOP)andChange(CHANGE)Thegoalofreectionislearningfrompreviousexperiences,whichmightinclude'anewwayofdoingsomething,theclaricationofanissue,thedevelopmentofaskillortheresolutionofaproblem'(Boudetal.
1985,p.
34).
Thisincludeslearningfromaproblem(singlelooplearning,codeS_LOOP)orlearningonamoregenerallevel(double-looplearning,codeD_LOOP)(cf.
ArgyrisandSchn1978).
Inaddition,itmayleadtochangesinpracticesandtheworkenvironment,whichwecodebyCHANGE(Daudelin1996;Moon1999).
(continuedonnextpage)HowDoesCollaborativeReectionUnfoldinOnline.
.
.
709justication)fromsuggestionsbasedonknowledge.
Therefore,datasetEincludesallinformationandcodesproducedfordatasetM(thewaycodeSUGwasusedfordatasetMisthesamethatSUG_KNOwasusedfordatasetE),datasetEiscodedslightlymorespecicregardingsuggestions.
However,thisdoesnotaffectthisanalysis,aswefocusonsolutionsbasedonexperiencesforreection.
Tworesearchersusedthecontentcodingschemetocodetherstdataset(datasetM)whichcontainedthreadedonlinediscussiondatainEnglish.
Codeswereassignedtoeachcontribution,andcoderswereaskedtomarkpartsofasentence,fullsentencesormultiplesentencesthatledthemtoapplyingthecode.
Thismeansthatmultiplecodescouldbeassignedtothesameunitofcoding(thecontribution),forexamplewhensomeonetalkedaboutownexperiences(EXP)whileincludingownemotions(EMO_OWN).
Thiswasdonefortworeasons.
First,thecomplexnatureofreectionmakesithardforcoderstocodeexactlythesamepartsofsentences:Forexample,whetheraphraseorapartofasentencebelongstoanexperienceornotistoosubjectivetobeformalized(seetheexamplesfromthedataasprovidedinthispaper).
Thisisalsonotcoretoouranalysis,aswewereinterestedinhowcontribu-tionsofusersmayinuencecollaborativereectionratherthanintheanalysisofconcretestatements.
Second,thereisnoneedforsuchformalizationorstricterrulesfortheunitofanalysis,asthiswouldmeanquantifyingreectiveutterancesbasedonthenumberoftextfragmentscoded.
Thiswouldbearbitraryandnotbackedbyliterature.
Toreducesubjectiveinterpretationsinthecoding,thecodersemployedastrictrulewhichstatesthatpiecesoftext(e.
g.
partsofasentence)canonlybecodedwithaspeciccodeifthecodercanpointatthewordsthatleadtoacode.
Thisway,forexample,onlytextpiecesinwhichsomeoneexplicitlymentionedthattheymadeaspecicexperiencewerecoded(codeEXP,referringtoownexperiences)andotherphraseswhichsoundlikeanexperiencewerediscardedasexplicitreferencestotheexperienceweremissing.
Usingthisrule,phraseslike'AfterinitiallystrugglingwithtakingcallsIsatdownwithmymanagerandtalkedthroughthedifferentvariousdifferentcallswewouldgetandhowbesttodealwiththem.
'(datasetM)werecodedTable3.
(continued)CodeDescriptionQuestions(Q_*)Questionsareimportantfacilitatorsincollaborativereection(WoerkomandCroon,2008;Zhu1996).
Questionsposedtorequestfurtherinformationfromothers(e.
g.
,togetabetterunderstandingofthesituation)aresupposedtoincreaseactivityinreectionandarecodedwithQ_INF.
QuestionsaskingforinterpretationsorviewpointsofothersaresupposedtoincreasethequalityofoutcomesinreectionandaremarkedbyQ_INT.
PrillaMichaeletal.
710asanexperience(EXP)becauseoftheirexplicitreferencetopastactivities('Isatdown(…)andtalked').
Likewise,phraseslike'Ithinkkeeping'todolists'foreachdayiseffectivethenjustworkthroughthemthroughouttheday.
Iwouldalsosuggeststayinglateandworkingharder.
'(datasetM)werenotcodedasrelatedtoownexperiences(althoughsoundinglikeit)becausetherewasnoexplicitmentioningofpastactivities.
Thisway,wemadesurethatpersonalinterpretationofpossiblethoughtsandintentionsinapieceoftextwaskepttoaminimum.
WeusedKrippendorff'salpha(HayesandKrippendorff2007)toassessintercoderreliabilityandtoensureahighdegreeofagreementbetweenbothcoders.
Aftercodingpartsofthecontent,theresearchersdiscusseddiffer-encesbetweentheircodingsinordertolearnabouttheirrespectiveunder-standingofthecontentcodingschemeandcommunicatetheirperspective.
Thiswasrepeatedafterallcontenthadbeencoded.
Afterdiscussing,bothresearchersreviewedtheircodingagain,andchangedcodesthatdidnottthenewunderstandingafterthediscussions.
Thisiscommonproceduretoensureintersubjectivityandthusreliabilityincontentcoding(e.
g.
,JohriandYang2017).
Forthenalcoding,Krippendorffsalphawasonaverage.
91acrossallcodes,witheachcodebeingabovethethresholdof.
66,whichisproposedbyKrippendorffastheminimumacceptableagreementforfurtheranalysis(HayesandKrippendorff2007).
AsdatasetEcontainstextwritteninaforeignlanguagenotspokenbytheresearchersuently1,twostudentresearchers,whoarenativespeakersforthislanguageanduentinEnglish,conductedthecontentcodingbasedonthecontentintheirnativelanguage.
Thiswasimportanttoensurethatmeaningsorstatementsarenotlostintranslation.
TheywerealreadyfamiliarwiththeprocessofcontentcodingandtheresearcherswhocodeddatasetMtrainedtheminapplyingthesamecontentcodingschemeusedforsetM,usingsamplesfromthatsettoensurethecodingprocesswasdoneinthesamefashion.
AfteranextensiveexplanationofthecodingschemethestudentscodedsampledatafromsetMtotrainapplyingthecodingscheme.
AfterwardstheircodingwascomparedtothecodingfordatasetMasdescribedaboveanddifferenceswerediscussed.
Thiswasrepeateduntilthecodingofthestudentswasconsistentwiththeinitialcoding.
Indoingso,itwasensuredthattheunderstandingoftheothercodersdidnotdeviatefromtheunder-standingoftheresearchers,andthatdatasetsMandEwerecodedinacomparableway.
AfterwardsthestudentscodeddatasetEfollowingthesameprocessasdescribedfordatasetM,resultinginKrippendorff'salpha(HayesandKrippendorff2007)of0.
67.
ItshouldbenotedthatweusedKrippendorff'salpha(HayesandKrippendorff2007)asameasurementofagreementtoensureahighdataqualityonly.
Fortheanalysis,weusedadatasetthatincludedonlythosecodeswhichwereassignedby1Seeaboveforreasonsofnon-disclosingthelanguage.
HowDoesCollaborativeReectionUnfoldinOnline.
.
.
711bothcoders.
Ontheonehand,thisledtoexcludingdataentriesbutontheotherhandtoahigherreliabilityofthenaldatasets.
4.
ResultsTobeopenforallpossiblewaysofhowcollaborativereectioncouldunfoldinourdata,weappliedseveralmethodstoanalyzethedata.
Thisincludedafrequencyanalysistocalculatehowoftenagivencodewaspresentatdifferentstagesofathread,thatis,howoftenacertaincodewaspresentintherst,secondandotherpostsinathread.
Thiswassupposedtoinformusontheroleofspecicutterancesinearlierorlaterstagesofreectionaspredictedbythereectionmodelspresentedabove.
Inaddition,welookedattherelationshipbetweenpostsandtheirpredeces-sors.
Takingintoaccountthatitsimmediatepredecessormaynotonlyinuenceapostandtoconsiderdifferentreadingstyles,wecalculatedcorrelationsbetweentheoccurrencesofcodesinapostandthecodesinallpredecessors.
Basedontheresultsoftheseanalyses,welookedforcausalrelationshipsbyusingsequentialpatternminingandregressionmodels(overviewinAppendixB).
4.
1.
Codefrequency:lookingforconversationowsCommonreectionmodelssuggestasequentialoratleastiterativeowofreection.
Therefore,weshouldseeadistributionofcertaincodesamongthephasesthatreectthisow(e.
g.
,experiencesshouldbeprovidedbeforesuggestionsbasedonexperi-encessetin).
Wethereforestartedwithafrequencyanalysis,computinghowoftenagivencodewaspresentatwhichpositionwithinapost.
IndatasetM,showninTable4,wecanseethatpeoplearemorelikelytorefertoexperiences(EXP)ratherthanlinkingtoknowledge(KNO).
Thedifferenceissmall,andthenumbersuctuate.
Criticaldiscussions(discussionscontainingdisagreements(DISAGR))arerareaswell,anddisagreementonlytendstohappenmoreofteninlaterstagesofdiscussions,whichmaybearesultofdisagreeingwithsuggestionsmade.
Agreement(AGR)startsslowlyindatasetMbutisbuildingupasathreadprogresses.
Wecanseethattheamountofagreementstatedrisesslightlywiththeamountofanswersinathread.
Thismaybeattributedtothesmallgroupsresponsibleforthedata:Agreementonexperiences,suggestionsandothercontributionsmayhaveledsomediscussiontobecomelonger,whereaslackofinterestorevendisagree-mentmayhaveledpeopletoleavethediscussionandthusstopit,giventhattherewereonlyafewpotentialparticipantsanyway.
Forsolutions,weseethatadvice(ADV)ismuchmorecommonthansuggestionsaccompaniedbyreasons(SUG),whichisdifferentfromdatasetE(seebelow).
ThismaybeattributedtothefactthatinthreeoffourgroupsofdatasetMsuperiorswereactiveinthegroupsandprovidedadvice.
Questionsaskingformoreinformationarelesscommonthanquestionsaskingforopinionsorinterpretation.
PrillaMichaeletal.
712Table4.
FrequencyanalysisofdatasetMinpercentages.
Itshowshowoftenagivencodewasappliedintherespectiveanswernumberwithinathread.
Post1istheinitialtopicwhichstartsathread.
Inthisdataset,wedidnotdifferentiatebetweensuggestionsbasedonexperienceorknowledge.
Thesecondcolumnshowshowoftenathreadreachedacertainlength.
Thelastrowshowshowoftenthecorrespondingcodewasassignedintotal.
Forexample,thistableshowsthattherstreplycontainedexperiencereportsin22%ofthecases.
BoldfacehighlightsdepictgurescharacterizingthedataorseparatingitfromdatasetE.
Post(#)EMO_OWNin%EMO_OTHin%INTin%EXPin%KNOin%DISAGRin%AGRin%ADVin%SUGin%S_LOOPin%D_LOOPin%CHANGEin%Q_INFin%Q_INTin%165171814222553522265691122982512253118350421414216241426464268122315128823511918189272791899965202020202020∑1719242712123362061281530HowDoesCollaborativeReectionUnfoldinOnline.
.
.
713IndatasetM,doublelooplearning(D_LOOP)andchangearereportedmoreoftenindatasetE,and(likeindatasetE)occurmoreofteninlaterstagesofthethreadasthediscussionprogresses.
FordatasetE,resultsofthisanalysisareshowninTable5.
ThreadsarelongerthanindatasetM,whichisduetothenumberofuserspergroup.
Wecanseethat(exceptforthesixthanswer(post#7)),experiencesarementionedinroughly40%ofallreplies.
ThenumbersdonotuctuateasmuchasindataM,andthedifferencebetweentheamountsofoccurrencesofthecodesisbigger.
Contrastingthistolinkingknowledgeasapossiblesolutiontosomeone'sissue,thecodeforlinkingknowledgewasusedlessoften(inaround20%ofallreplies).
WecanthereforenotethatinthediscussionsindatasetE,referencingexperiences(EXP)wasmorecommonthanlinkingknowledge(KNO).
Asagreement(AGR)anddisagreement(DISAGR)indatasetEaretheonlycodesinthecodingschemetomarkinteractionwithinatopic,wecanseethattherewerenotmanycriticaldiscussions:Theamountofcodefordisagreement(DISAGR)isquitelow,thoughincreasingasthediscussionevolves.
Agreement(AGR)appearedmoreoften,indicatingthattheusersengagedindiscussion,thatthecontributionswereperceivedashelpful,andthatuserscouldrelatetothediscussion.
AsindatasetM,theamountofquestionsaskingforopinionsorinterpretation(Q_INT)ishigherthanforquestionsonfurtherinformation(Q_INF),andthesequestionsareaskedthroughoutthethreads.
Thissuggeststhattherewasinterestincontinuingthediscussionandthatthesequestionsmayhaveaffectedit.
Whenitcomestosolutionorientation,usersprovidedlittleadvice(ADV),butrathertriedtoprovidesuggestionswithreasons(SUG).
Thehighpres-enceofexperience-basedcontributions(EXP)isalsovisibleinthesugges-tions,astheoverallfrequencyofexperience-basedsuggestions(SUG_EXP)ishigherthantheoccurrenceofknowledge-basedsuggestions(SUG_KNO).
Whatisstrikingisthatthedatashowsthatafterthethirdreply,suggestionsbasedonknowledgevanished,whichunderpinsthefocusonexperienceexchangeindatasetE.
Onlyafewthreadsreachedastateinwhichsomeoneindicatedthattheylearnedsomething,andtheoveralloccurrenceofquestionsaskingformoreinformationisratherlow.
However,thecodesindicatinglearningonlystarttoappearafterthesecondanswergiven(post#3),whichindicatesthattheybuildonwhatwasexchangedbefore.
Inaddition,thelownumberofcodesconcerninglearningdoesnotmeanthatlearningdidnothappen:Theusersofthesystemthatwemetduringthestudytoldustheyhadtakenawayvariouslearningsfromtheirinteractionwiththeothersinthesystem,buthadnotdocumentedit.
Besidesenablingacomparisonbetweenthedatasets,thisanalysisshowsthattheelementssuggestedbyreectionliteraturewerepresentinourdata,andthatthereforewecancallthecorrespondingdiscussionsreective.
PrillaMichaeletal.
714Table5.
FrequencyanalysisondatasetEshowshowoftenagivencodewasappliedintherespectiveanswernumberwithinathreadforthreads,whichreceivedatleastonereply.
Allfrequenciesarepercentages.
Post1istheinitialtopic,whichstartsathread.
Thesecondcolumnshowshowoftenathreadreachedacertainlength.
Thelastrowshowshowoftenthecorrespondingcodewasassignedintotal.
ThecodeEMO_OWNisnotshown(3%ofthethirdreplycontainedownemotion).
Forexample,thistableshowsthattherstreplyindiscussioncontainedin43%ofthecasescodesforexperiencesandthat35threadsreachedthislength.
BoldfacehighlightsdepictgurescharacterizingthedataorseparatingitfromdatasetM.
Post(#)EMO_OTHin%INTin%EXPin%KNOin%DISAGRin%AGRin%ADVin%SUG_EXPin%SUG_KNOin%S_LOOPin%D_LOOPin%CHANGEin%Q_INFin%Q_INTin%135314462361733360235114317326926146314329101745212810341437424133829292113844452255451452718235561955421111163255117161913191319619812883333332517889911443311331133111071443141414141411425502525∑20269443545155414564746HowDoesCollaborativeReectionUnfoldinOnline.
.
.
7154.
2.
Clusteranalysis:focusingontypesofthreadsWhilethefrequencyanalysisdidnotshowaclearsingularowofcollaborativereection,wefoundthattherewasaninnerdifferentiationofthreads:Forexample,wefound'experience-heavy'threads,whichcontainedahigheramountofpostsincludingexperiences(codeEXPfromTable7)thanothers,and'knowledge-heavy'threadsthatincludedmorepostsbasedonknowledge(KNO).
Asdescribedabove,reectionliteraturesuggeststhattheprovisionofexperiencescreatedmorereectionthantheprovisionofknowledge,thatadviceisinferiortosuggestionsbasedonexperienceswhenreectionisthegoals,andthatlearningisthepositiveoutcomeofsuccessfulreection.
Usingtherelativefrequencyofcodesinadiscussionthread,wecreatedclustersinthedataforthesedistinctionstoanalyzewhetherreectionwentdifferentlyine.
g.
threadsheavilyrelyingonexperiencesasopposedtothreadsrelyingonknowledgeprovided.
Tocreatetheclusters,wecomputedtherelativefrequencyofacodeinathreadbydividingthenumberofoccurrencesofthecodebythenumberofitsoveralloccurrencesinthedataset.
Wethencharacterizedathreadas'heavy'foracodeifthecodeappeared1.
5times(usingthisfactorasaheuristic)moreoftenthanonaverage.
Notethatthiswaythesamethreadisassignedtoaclusterineachclustering.
Basedonthedifferentelementspresentinreection,wecreatedthefollowingclusterings:&Experience-KnowledgeClustering:DependingonwhetherthethreadwasdominatedbyExperienceorKnowledgethethreadswereassignedtothecorrespondingcluster.
Ifneitherthecodesforexperience(EXP)norknowledge(KNO)weredominanttotheextentdescribedabove,threadswereassignedtoanUndecidedcluster.
&Advice-SuggestionClustering:ThreadswereputintoanAdviceclusterorinaSuggestioncluster(unitingcodesSUG_EXPandSUG_KNOinordertocomparedatasetsEandM),andagainifneitherwasoverwhelminglyrepresented,threadswereassignedtoanUndecidedcluster.
&LearningClustering:Wecreatedtwoclusters.
TherstclustercontainedallthreadsthatincludedLearningorthewillingnesstochange(unitingcodesS_LOOP,D_LOOP,orCHANGE)andanotherclustercontainingallthreadsnotreachingthisstatus(No-Learning).
Here,wedidnotusetheweightingofcodefrequencydescribedabove,asgiventheoverallnumbersoflearningrelatedcodesfoundinthedata,weassumedthebinarydistinctionwhetherathreadcontainslearningtobesufcienttocreatetheclusters.
Withintheseclusters,weperformedacorrelationanalysistoderivepossibleindicatorsofwhichtwocodesmighthavearelationshipwithintheanalyzedcon-versations.
AnoverviewoftheclustersisshowninTable6.
AscanbeseeninTable6,theclustersdifferedinsize.
Asaresult,weexcludedtheAdvice,KnowledgeandUndecided(Experience-Knowledge)clusterfordatasetEintheanalysis,asthePrillaMichaeletal.
716numberofpostsassignedtothoseclustersweretoosmall.
OtherclustersliketheKnowledgeclusterindatasetMareborderlineacceptable.
FromthesenumbersinTable6,wecanalreadyderivethatindatasetEexperienceexchangedominatedwhereasthediscussionsindatasetMwerenotheavilyfocusedoneitherexperienceorknowledge.
Intermsofwhetheradvice(ADV)orsuggestions(SUG)dominated,wecanseethatthethreadsindatasetMareverybalanced,andthatindatasetEmostthreadswerefocusedonsuggestions.
Concerninglearning,wecannotethatroughlyhalfofthepostsindatasetEbelongedtothreadsthatreachedastateoflearning,whereasindatasetMthisnumberisclearlybelowhalf.
Weperformedindependentt-teststocomputewhetherthethread-lengthvariesfromclustertoclusterineachsetofclusters.
Althoughtheaveragesofthreadlengthsdifferfromclustertocluster,wefoundthatthereisnosignicantdifferenceintermsofthread-lengthamongtheclustersinallsets.
4.
3.
InuencingvariableswithinathreadWecomputedcorrelationsbetweencodesinacurrentpostandcodesinpreviouspoststoevaluatetowhatextentcertaincodesarerelatedtoeachother.
Toreectdifferentreadingbehaviors,wecomputedonesetofcorrelationsbasedoncodesassignedtoapostandallpreviouspostsinthesamethread,andinanotherset,wecomputedonlythecorrelationbetweenthecodesinapostinrelationtothecodesintheimmediatepreviouspost.
Thisreectstwodifferentassumptionsonhowpeoplewouldcontributetoareectiveonlinediscussion:therststyleencompassespeoplewhoreadanentirethreadbeforephrasingananswerandthesecondstyledescribesuserswhofocusonthelastpostbeforephrasingananswer.
Inthissectionwereportcorrelationsbetweentwocodesfoundinbothanalyses,includingregressionmodels.
Table6.
.
Clustersizesforthedifferentdatasets.
Thetableshowsthenumberofthreadsassignedtoeachcluster(#T)andthetotalnumberofpostsineachthread(includingtheinitialpost;#P).
TheclustersAdvice,Knowledge,andUndecided(Experience-Knowledge)havebeenremovedduetohavingatoosmallsize.
ClusteringClusterDatasetMDatasetE#T#P#T#PExperience-KnowledgeExperience124919133Knowledge1037(6)31Undecided43137(7)38Advice-SuggestionAdvice2374(3)23Suggestion187116118Undecided24781361LearningLearning187212109No-Learning471512093HowDoesCollaborativeReectionUnfoldinOnline.
.
.
7174.
3.
1.
ProcedureofanalysisDuetotheveryhighnumberofcodesinthedatasets,wewereverylikelytoobservecorrelationsinthedatasample.
Wethereforeonlylookedatcorrela-tionswithaneffectsizeof0.
2andabove.
Inbothdatasetsthiseliminatedroughlyhalfofallcorrelationsfound.
Inaddition,weremovedallcorrelationsofcodeswithoutareasonableexplanationofhowtherespectiveutteranceswouldinuenceeachotheraswell.
Forexample,wefoundarelationshipbetweendoublelooplearning(D_LOOP)andreportsaboutotherpeople'semotions(EMO_OTH).
Thisandothercorrelationswerelikelytohaveoc-curredbychanceandsomelownumbersofcodeassignmentswithinthedatasets(cf.
Table4andTable5).
Inthefollowingparagraphs,wereportoncorrelationsfoundbetweencodeswhichwerepresentinmultipleclusters,indicatingthattheircorrelationmightholdtrueacrossvariouscircumstanceswithinathread.
Thisincludescorrelationsfrombothcalculations(allpredecessorsandonlytheimmediatepredecessor).
Forthesakeofsimplicity,wereportboththeweakestandthehighesteffectsizeofthecorrelationinquestion.
Additionally,weconductedlinearregressionanalysisforthestrongestcorrelationoftwocodes,whichisreportedtogetherwiththecorrelationanalysis.
Wealsoremovedallcorrelationswhichhadalownumberofoccurrencesinthedatasets.
Forexample,wefoundahighcorrelationbetweenchallengingsuggestions(DISAGR)andagreement(AGR),butdiscardeditasitoccurredonlyonce.
Foracomprehensiveoverviewofallkeptndingsrefertotheappendix.
NotethatwhenwereportonthecorrelationofcodeAandcodeB,wemeanthatBoftenoccurredafterA.
4.
3.
2.
DatasetMWeobservedthreecorrelationsinmultipleclustersshowingthatmentioningexperience(EXP;minr=0.
247,p<0.
001;maxr=0.
435,p=0.
002;maxr2=0.
189,F=10.
978,p=0.
002),knowledge(KNO;minr=0.
211,p=0.
001;maxr=0.
329,p=0.
004;maxr2=0.
183,F=10.
551,p=0.
002)anddoublelooplearning(D_LOOP;minr=0.
309,p=0.
008;maxr=0.
557,p<0.
001;maxr2=0.
310,F=32.
367,p<0.
001)correlatewithagreement(AGR).
Thisshowsthatusersengagedwithcontentcontributedbyothers,whichisaprerequisiteforcollaborativereectiontohappeninthemodelspresentedabove.
Thisissupportedbythemoderatetogood(D_LOOP)explanatorypoweroftheregressionmodelswecomputed.
ThefollowingsequencefromdatasetM2exempliesthisbyshowinghowoneusercontributedanexperienceandtheotherreferstoitwithagreement,thusre-assuringtheoriginalcontributorintheiraction:'(…)IwassooterriedtotellmymanagerIneededdaysoffformymanager.
However,whenItoldhershewassoshockedandhappyforme.
Shespreadthe2AllexamplesfromdatasetMaredirectquotescontainingoriginalorthography.
PrillaMichaeletal.
718newstoalmosttheentireofceanddidntaskmetomakeupthetime.
Iguessourmanagersarecool:)'(CodeEXP)'Welldone,itsgoodthatyoutoldhimatleastit'llgivehimtimetoprepareandaalessshocklol'(CodeAGR)Ourdataindicatesthatquestionsmayalsohaveaninuenceonhowadiscussionthreadunfolds.
Questionsforopinionsorinterpretation(Q_INT)correlatewiththeprovisionofexperience,andexplainupto27%ofthevarianceofcodeforexperienceinthedata(EXP;minr=0.
233,p=0.
04;maxr=0.
442,p<0.
001;maxr2=0.
266,F=12.
697,p=0.
001).
Fromthis,wemayinterpretthatusersanswerthosequestionswithaviewpointbasedontheirownexperience.
WeobservedthiscorrelationintheclustersLearningandUndecidedforboththeExperience-KnowledgeclusteringaswellastheAdvice-Suggestionclustering.
Thismayindicatethatansweringques-tionsforinterpretationwithownexperiencesmaybebenecialforthreadwithregardtolearningasanoutcomes.
Atypicalexampleisshownbelow:'Howdidyoundthesituationanddidithelp'(CodeQ_INT)'Heisalwayssayingthathecancopebuthehastorealisethathe'snotastashewere,hewillenduphavingafall'(CodeEXP)Therearenocodeswhichdirectlycorrelatewithsinglecodesforlearning(S_LOOP,D_LOOPorCHANGE).
Thismaybeattributedtothelownumberoflearningcodesoverall,which(asmentionedinsections3.
3and4.
1)wasalsoobservedinpreviousstudies.
Therefore,wedecidedtofocusonanalyzingonanykindoflearningoutcome,takingintoaccountthatsingleanddoublelearningaswellaschangearealldesirableoutcomesofcollaborativereectionandcannotbepreferredovereachother.
Forthecorrespondinganalysis,wecom-putedanewvariableastheunicationofthethreelearningvariablesandcalleditLEARN.
Inacorrelationanalysis,wethenfoundcorrelationsofownemotions(EMO_OWN;r=0.
414,p<0.
01),experience(EXP;r=0.
564,p<0.
01)anddisagreement(DISAGR;r=0.
417,p<0.
01)withthisvariable.
Alinearregres-sionanalysisshowsthattheoccurrenceofthecodeformentioningownexperi-ence(EXP)explains32%ofthevarianceofLEARN(r2=0.
318,F=102.
829,p<0.
01)andthatthemodelgetsbetterwhenaddingthecodeindicatingownemotion(EMO_OWN)andthecodeforchallengingexistingsuggestionsoropinions(DISAGR;r2=0.
412,F=51.
065,p<0.
01).
Thissuggeststhatsharingexperienceshasapositiveimpactonlearningdocumentedasaresultinthreads,andthatthearticulationofownemotionsanddisagreementamplifythiseffect.
Interestingly,singlelooplearning(S_LOOP)andchange(CHANGE)correlatewithsuggestions(SUG;minr=0.
265,p=0.
002;maxr=0.
360,p=0.
002;maxr2=0.
315,F=16.
081,p<0.
001)andexperiencereports(EXPminr=0.
256,p=0.
028;maxr=HowDoesCollaborativeReectionUnfoldinOnline.
.
.
7190.
456,p<0.
001;maxr2=0.
307,F=33.
615,p<0.
001)respectively.
ThosecorrelationswereobservedintheclustersbasedontheimmediatepredecessorcalculationintheclustersforLearning,SuggestionandKnowledge.
Thecorrespondingregressionmodelsshownthatboththecodeforsingelooplearning(S_LOOP)andthecodeforintendedorplannedchange(CHANGE)explainonethirdofthevarianceintheoccurrenceofthecodeforexperiencereports(EXP),whichsupportsthisinterpretation.
Thismaysuggestthataddingownexperiencesorknowledgetodocumentedlearninghelpspeopletomakesenseofthislearningandpotentiallyadoptit.
4.
3.
3.
DatasetEWeobservedthatadvice(ADV;minr=0.
236,p=0.
006;maxr=0.
552,p<0.
001;maxr2=0.
103,F=6.
740,p=0.
012)correlateswithsuggestionsbasedonownknowledge(SUG_KNO),suggestingthatusersmightnotbecontentwithplainadviceandthusaretryingtoprovidemorereasoning.
Inaddition,suggestionsfollowingonadvicearenotbasedonexperience(SUG_EXP)butonknowledge(SUG_KNO),whichmayindicatethatadviceleadsathreadintoadirectionnotdesirableforcollaborativereection.
ThosecorrelationswereobservedmainlyintheUndecidedclusteroftheAdvice-Suggestionclustering,aswellastheLearningcluster.
Itshouldbenoted,however,thatcorrespondinglinearregressionmodelsstayedatlowexplanatorypower,andthatthereforefurtherworkisnecessarytoinvestigatethiscorrelation.
Theexample3belowillustratesit:'Hi.
Yourpriorworkisveryusefulbecauseitmakesiteasiertomakesenseofthelawsandtheactualapplicationofthelaw.
Youshouldofcoursereadeverything.
Muchsuccessandkindregards.
'(CodeADV)'Heythere,(…)Whenitcomestothewrittenpart,itwouldbegoodtogooverthelawsatleastonce,marktheimportantchapterssothatyouknowexactlywhereeverythingisbecauseyouwillhave4open-endedquestionsandwillhavetodescribe/writeacertainarticle(…)'(CodeSUG_KNO)Suggestionsbasedonexperiences(SUG_EXP;minr=0.
220,p=0.
011;maxr=0.
326,p=0.
001;maxr2=0.
106,F=12.
716,p=0.
001)correlatewithconsecutivesuggestionsbasedonexperienceintheLearningandExperiencecluster,showingthatsomethreadsrevolvearoundmultiplesuggestionsfrompersonalexperience.
Thismeansthatmakingsuchsuggestionsmaybeatriggerforadditionalsuggestions,leadingtoaconversationinwhichpeopleexchangetheirpracticesandsuggestthemtoothers(seetheexamplebelowforatypicalconversation).
Theregressionmodels3AscontentfromdatasetEwascreatedinadifferentlanguage,examplesgivenfordatasetEaretranslationsoforiginalcontributionscreatedforthispublication.
Assuch,somedetailsmaybemissingorchangedbecauseofthetranslation.
PrillaMichaeletal.
720wecomputed,however,remainatlowexplanatorypower,andsofurtherworkneedstobedonetolookatthisrelation.
'IsendthelatestinformationandeventsattheEmploymentServicetomyclientsapproximatelyonceamonth.
Clientswelcomethistypeofinformationprovisionandareverysatised.
Asrecentlyasthisweek,aclienttoldmethatifshehadnotreceivedsuchinformation,shewouldhavemissedoutonalot(…)'(CodeSUG_EXP)'Ihavecreatedamailinglistusing[systemA],and[systemB]has(nally)addedtheclients'e-mailaddresses.
[anumber]havesubscribedforthismailinglistandInowsendtheme-mailsthatareformulatedbetter.
(…)EveryThursday,Icheckwhohasbeenaddedtomyrecordanew(…),Iaddthemtothelistandnotifythemofthis(andsendthemthelatestnews).
'(CodeSUG_EXP)Atthesametime,suggestionsbasedonexperiences(SUG_EXP;minr=0.
232,p=0.
001;maxr=0.
477,p<0.
001;maxr2=0.
227,F=34.
125,p<0.
001)nega-tivelycorrelatewithquestionsforopinionsorinterpretations(Q_INT),hintingthatthosesuggestionsalreadyprovideenoughexplanationsothataskinganadditionalquestionaswellasfurtherdiscussionisnotneeded.
ThosecorrelationswereobservedintheLearning,SuggestionandExperiencecluster.
4.
4.
TheInuenceofimmediatepredecessorsinthreadsThepreviousanalysiscoveredcorrelationspresentinthecalculationsforallprede-cessorsandfortheimmediatepredecessor,showingwhatpossibleinuencesexistthroughoutthediscussion.
Nowwefocusonspecicresultsforimmediateprede-cessors,whichbringforwardinterestinginsights.
4.
4.
1.
DatasetMAswesawearlier,conversationsintheonlinediscussionsdonotstopwhensomeonementionssomethingwhichcanbecodedaslearning(S_LOOP,D_LOOPorCHANGE).
Whenreferringtoonlyimmediatepredecessorswecanobservethatdoublelooplearning(D_LOOP)mayhaveinuencedthesubsequentpost,asinthosepostsexperiencereports(EXP;minr=0.
250,p=0.
034;maxr=0.
296,p=0.
039;maxr2=0.
194,F=17.
382,p<0.
001),linkstoknowledge(KNO;minr=0.
254,p<0.
001;maxr=0.
301,p=0.
01;maxr2=0.
091,F=6.
985,p=0.
01)andevenchange(CHANGE;minr=0.
205,p=0.
002;maxr=0.
388,p=0.
001;maxr2=0.
151,F=12.
781,p=0.
001)canbeidentiedoften.
Thosecorrelationswerefoundintheunclusteredcorrela-tionanalysisaswellasintheLearningandAdvicecluster.
Whiletherelationbetweenlearningandchangeisnotverysurprisingbutdesirableinreection,theothertwocorrelationsmayindicateaninterestingpattern,astheysuggestthatrelatingastatementaboutlearningtoownexperiencesorknowledgemayHowDoesCollaborativeReectionUnfoldinOnline.
.
.
721easetheadoptionofthislearningforoneself.
Themoderateexplanatorypowerofthelinearregressionmodelsforthecodeforexperiencereports(EXP)backsthisinterpretationup.
Thisndingresemblesourndingsforthecodesforsinglelooplearning(S_LOOP)andplannedrespectivelyintendedchange(CHANGE)indatasetMasreportedabove.
ThefollowingexamplefromdatasetMunderpinsthis:'Itishardtodealwithphonecalls.
Sometimesitmakesmeanxious,butthebestwayIhavefoundouttodealwiththemishavingalistofFAQsandalistofcontactstowhomIcantransferthetrickycalls.
CodeD_LOOP)'ThelistofFAQ's/usefulcontactsisactuallyreallyuseful-Ican'tthinkoftheamountoftimesIhavepickedupacall,thenwastedtimetryingtondthecorrectprocedure-willdenitelybeusingthat.
'(CodeEXP;CodeCHANGE)4.
4.
2.
DatasetEWecanobservefeedbacksequencesofagreement(AGR;minr=0.
204,p=0.
05;maxr=0.
381,p=0.
002;maxr2=0.
145,F=9.
998,p=0.
002)correlatingwithagreement.
ThisshowsthatindatasetEtherewasalotofpositivereinforcementthroughoutthediscussions.
ThesecorrelationsappearedintheNo-LearningandbothUndecidedcluster.
'[name]youradvicewasappropriate.
Itisimportantforherbodylanguagetobecorresponding–i.
e.
thatthethingssheissayingarealsodemonstratedbyherposture+openposture,eyecontact,justasyouadvisedher.
(…)'(CodeAGR)'(…)justaswetalkedthelasttime,Iagreewith[name],namelythatyouradvicewasappropriate–thatherpostureisopen,eyecontactandbeingatease(situationpermitting)CodeAGR)Wealsoobservedonepossibleinuenceonlearning:Agreement(AGR;minr=0.
201,p=0.
004;maxr=0.
286,p=0.
003;maxr2=0.
154,F=10.
747,p=0.
002)correlateswithsinglelooplearning(S_LOOP).
ThishappenedintheLearning,butalsointheUndecided(Advice-Suggestion)cluster,andespeciallytheexplanatorypowerofthelinearregressionmodelforagreement(AGR)suggeststhat–astrivialasitseems–positivereinforcementmayfosterlearningfromcollaborativereection.
4.
5.
TheinuenceofallpredecessorsinthreadsTherewasonlyonecorrelationtobefoundonlyforallpredecessorinathread.
IndatasetEwefoundthatquestionsforinformation(Q_INF)negativelycorrelatedwiththeprovisionofexperience(minr=0.
284,p=0.
003;maxr=0.
302,p=0.
018;maxr2=0.
091,F=5.
916,p=0.
018).
ThissuggeststhataskingforfurtherinformationhindersPrillaMichaeletal.
722experiencestobearticulated,whichsoundsreasonable:Usually,suchfurtherinforma-tionmayclarifyissues(probablyincludingexperiences),butisusuallynotmadeupbyexperiences.
Thisrelationship,however,isonlyweaklysupportedbytheregressionmodelswecomputed,andthereforeitneedsfurtherinvestigation.
4.
6.
TheinuenceonanswersprovidedtoathreadInordertoevaluatewhatatopicneedstoprovidetoreceivereplies,wecomparedthemeanvaluesforcodesinthetopicsthatreceivedareplyandthetopicsthatdidnotreceiveareply.
Wefoundasignicantdifferenceintheoccurrenceofadvice(ADV)betweenthetopicswithreplies(M=0.
25,SD=0.
44)andthosetopicswithoutreplies(M=0.
52,SD=0.
51)inanindependent-samplest-test(t(47.
575)=2.
105,p=0.
041).
Thissuggeststhatthemoreusersareincludingplainadviceintheinitialtopic,thelesslikelytheuseristoreceiveareplytothediscussion.
Thissuggeststhatprovidingadviceinitiallyisnotdesirableforonlinediscussionsingeneralandforonlinecollaborativereectionspecically.
Theindependent-samplet-testdidnotshowsignicantdifferencesforothercombinationsofcodes.
4.
7.
PatternminingInordertolookdeeperintosequencesofcodeandwhattheymaymeanfortheowofcollaborativereection,weperformedananalysiswithsequentialpatternminingalgo-rithmsinordertouncovercommonsequencesinusers'postingactivitythatmayindicatereectivelearning.
Usingsequentialpatternmining–inparticularthePrexSpan(Peietal.
2004)andSPADE(Zaki2001)algorithms–weattemptedtoidentifythreadsconsistingofthesamepatterns,whichmeansthattheyincludecausalrelationshipsoftypesofstatements,whichcouldserveasindicationsofsuccessfulreectivelearning.
However,wedidnotndanymeaningfulpatternsotherthantheonesconrmingthecorrelationanalysis(presentedinsection5).
Onereasonforthismaybethelevelofdetailofthecodingscheme(manydifferentcodes)incombinationwiththesizeofthedataset.
Thatis,wemayneedmoredatainordertobeabletodetectmeaningfulcodingsequences.
Weplantofurtherinvestigateonthisresearchlineinfuturework.
5.
DiscussionOuranalysisofdiscussionthreadsindifferentcasesprovidesinitialinsightsintohowreectivediscussionsunfoldinonlinecommunitiesinworkplacesettingsandwhichelementsofdiscussionsfosterorhindercollaborativereection.
Weareawareofthefactthatgiventhesizesofourdatasetsandscarcecausalrelationships,ourresultscannotbegeneralized,andthatthereisaneedtofurtherinvestigatethem.
However,theresultsshowthatthereisadiversityinonlinecollaborativereectionthatisnotpredictedbycommonmodels,andthatreectionisnot'messy'butfollowscertainpaths,whichhavenotbeenidentiedanddescribedfully,butcharacterizeHowDoesCollaborativeReectionUnfoldinOnline.
.
.
723collaborativereection.
Therefore,ourworkbringsforwardnewinsightsintohowreectionunfoldsinonlinecommunities.
Ratherthanderivingrulesfortheowandsupportforcollaborativereectionfromourresults,wederivehypothesesandsuggestionsfromourndings,whichneedtobeevaluatedinsubsequentstudies.
5.
1.
Comparingresultsfromthedatasets:themultiplepathsofcollaborativereectionWhencomparingthecorrelationsforbothallandimmediatelyprecedingcommentsinthetwodatasetsMandE,theyseemtodifferfromeachotherverymuch.
Bothcontainreectivecontent,buttheowofthethreadsseemstobedifferent.
Weattributethistotheusergroupsandtoolshavingslightlydifferentpurposes,differentgroupsizes,slightlydifferentdomains,differenttimespansofusageandpossiblyalsodifferentculturesintheworkplaces(seeTable2).
Reectivediscussionsmayunfolddifferentlyindifferentcontexts,andpotentiallysupportneedstobeadaptedaccordingly.
Forexample,inthesmallergroupsreectingtogetherindatasetM,peoplemighthaveputmoreemphasisindocumentingtheirlearnings,astheymighthavefelttheyhadcloserrelationshipstoeachother.
Asanotherexample,thecultureoftheorganizationdatasetEwascreatedinwasratherlittlefault-tolerant,whichmayhaveledtolesspeoplesharingexperiences.
DatasetMcontainsmorecodesthatpointtothedocumentationoflearning.
Inaddition,wefoundrelationsbetweenthecodesrelatedtolearning(S_LOOP,D_LOOP,andCHANGE)andfollow-upexperience-andknowledgeexchange.
Thismaybeattributedtothefactthatinthedataset'sfourgroups,inwhichpeopleknoweachother,theparticipantsworkedtogethercloselyandthereforediscussedissuesin-depth,evencontinuingafterinitiallearningsuccesswasdocumented.
IncomparisontodatasetM,datasetEcontainsalotofsuggestions–halfoftheclusterscreatedaresuggestion-heavy,andmostcorrelationsfoundincludesugges-tionsbasedonexperiencesorknowledge.
Therewasnoindicationofwhatmayhavecausedsuggestionstoappearofteninthediscussions.
Thisfocusonsuggestionsmaybeattributedtothesizeofthegroup,whichwasconsiderablylargerthanindatasetM.
Moreparticipantsmeanmorepotentialideasfordealingwithcertainchallenges.
Atthesametime,incontrasttosmallergroups,thereisanincreasedlikelihoodthatanissuesharedwith200(andmore)peopleisreceivedbysomepeoplewhohavealreadymadesimilarexperiencesandcanprovidetheirsolutions.
Thus,ratherthanengaginginsensemakingoftheexperience,thesepeoplemayhaveprovidedsolu-tionsthatworkedforthemrightaway.
Thesedifferencesshowthediversitythatreectionprocessesmaydemonstrateinpractice,andhowgroupsizeandsettingmayinuencetheoverallwayofreectingtogether.
Thismayberegardedasoneofthecentralndingsourworkpointsto:Ratherthanbeingmessyorstrictlyadheringtopathsshowninmodels,collaborativereectionmayfollowdifferentpaths,whichareinuencedbyfactorssuchasgroupsize,familiarityamongpurposeofthereection,culturalcontext,andothers.
WhilePrillaMichaeletal.
724thismaynotseemsurprisingonrstsight,withtheexceptionofdeGrootetal.
(2013),ithasrarelybeendiscussedinresearchoncollaborativereection.
OurworkresonateswithdeGrootetal.
(2013)andaddsonit:Totheknowledgeoftheauthors,thisistherstinvestigationofonlinecollaborativereectioncontentthatshowsthattherearemultiplepathsandpresentscertainsequencesofreectionthatdescribe(partsof)thesepaths.
Whileourworkcannot(anddoesnotaimto)presentanexhaustivelistofpathsthatcollaborativereectionmaytake,itshowstheneedtoinvestigatethesepathsinfutureworkfortheimplementationofpropersupportforcollaborativereection.
5.
2.
ElementsofreectionandtheirrelationsInthissectionwediscussourndingsinmoredetailtoanalyzeandinterpretthedifferentsequencesandpathswefound.
5.
2.
1.
ExchangeofexperienceInbothdatasets,weobservedthatthediscussionthreadsoftencontainexperiencereports(andmoreexperiencereportsthanlinkstoknowledge).
Thisisinlinewithreectionliterature(Boudetal.
1985;Schn1983)andhighlightsthatthethreadsshowexperienceexchangeaspartofreectiveinteraction.
Wealsoobservedthatsetsreportingonexperiences(EXP)correlatewithothersagreeingwiththeperspectivearticulatedinthis(AGR)forbothdatasets,whichshowsthereciprocityofcollab-orativereectionasdiscussedinsection2.
2.
Thissuggeststhatpeopleoftenfoundexperiencessimilartotheirowninthisexchange,andthattheyrelatedtothese.
Italsoindicatesthatpeoplediscussedina'healthy'environmentinwhichcolleaguesaresupportingeachotherwhiletalkingaboutissuesandideas.
Ingeneral,havingotherpeoplepresentishelpfultodiscussissuesandtoreceivefeedbackonideas(FleckandFitzpatrick2010;Raelin2002).
Wefoundatleasttwospecicrolestheexchangeofexperiencestookinthedatasets,amongwhichoneisinlinewithwhatcouldbeexpectedfromtheliterature,ofwhichonerepresentsaninterestingandrathersurprisingnewinsight.
First,wefoundthattheoccurrenceofcodeforexperiencereports(EXP)explains32%ofthevarianceoflearningoutcomesdocumentedinathreadfordatasetM,withadditionalcodesforpersonalemotion(EMO_OWN)anddisagreement(DISAGR)raisingtheexplanatorypowerto41%.
Thisisagoodexamplefortheneedforarticulationofexperiencesincollaborativereection,which,inlinewithreectionliteratureasdescribedabove,suggeststhattheexchangeofexperienceisastrongfactorforthesuccessofcollaborativereectionandneedstobesupportedexplicitly.
Second,andmoresurprisingly,wefoundcodesforexperience(andknowledge)tooccurafterlearninghadbeendocumentedinathread,whichisnotmentionedinliteratureon(collaborative)reection.
Lookingcloserattherespectivecorrelation,itmakessenseforcollaborativereection,asrelatingthelearningandchangedocumentedbyotherstoone'sownexperiences(orknowledge)helpstomakesenseofthesolution,toHowDoesCollaborativeReectionUnfoldinOnline.
.
.
725decideonitsapplicabilityforoneselfortoapplyit.
Thiscouldmeanthatfacilitationmechanismscancontinuetosupportusers'reectionevenafterlearningwasdocumentedandcanencourageuserstorelatetheirexperiencestothelearningdocumented.
5.
2.
2.
ProvisionofsuggestionsbasedonexperiencesOuranalysisontheoccurrenceofsuggestionsbasedonexperiencesandwhatmayhavefosteredorcausedthemdidnotprovidetheresultsweexpected.
Instead,theinterestingndinghereliesinwhatwedidnotnd:Lookingatreectionmodelsandreectionliterature,weassumedthattheprovisionofsuggestionsbasedonexperi-ences(SUG_EXP)shouldbepositivelyinuencedbytheprovisionofexperiences(EXP),andthattheprovisionofknowledge(KNO)shouldhavesimilareffectsontheprovisionofsuggestionsbasedonexperience(SUG_KNO).
However,wedidnotobserveanyoftheserelationshipsinourcoding.
Instead,weobservedthatsugges-tionsaremadefromthebeginning,thatis,withoutanyclearprecedingtypeofstatement.
Additionally,wefoundthatforsomethreadslearningprecededtheprovisionofsuggestionsbasedonexperience(datasetM)andforothers,suggestionsbasedonexperiencesfollowedeachother.
Thisiscounterintuitiveandsurprising,anditsuggeststhatreectionmaygodifferentwaysthansuggestedintheliterature.
Whatwemayderivefromthisisthattheprovisionofsuggestionsmaybecausedbydifferentelementsofaconversationandcanhappenanywhereinathread,andthatitmaybeworthwhiletoencourageuserstoprovidethemfromthebeginningon.
Itshouldbenoted,however,thatsuchstatementsofsuggestionsbasedonexperi-encesbydenitionincludestatementsonownexperiencesasshownintheexamplesprovidedinthispaper(seee.
g.
secondexampleinsection4.
3.
3andAppendixTable7).
Nevertheless,forthesupportofcollaborativereectionthismeansthatratherthanfosteringaprocessofexchangingexperiencesandthenderivingsugges-tionsfromthem(asdescribedinalmostallmodels),mechanismsmayfostertheprovisionofsuggestionsbasedonexperiencesanytimeandwithoutnecessaryprecedingelementsoftheconversation.
5.
2.
3.
Theroleofquestions:fosteringquestionsforinterpretationOneofthecommonfactorsinreectionmodelsarequestionstobeaskedinordertostructure,moderateandleadreectiontosuccess.
Inourcodingscheme,webuiltonZhu(1996),whofoundthatwithrespecttotheirinuenceonreection,thereisaneedtodifferentiatebetweenquestionsforfurtherinformationandquestionsforinterpretationofexperiences,asthelatterstimulatereectionwhiletheformerdonot.
Whilewecannotprovethisnding,ourdatasuggeststhatthisdifferentiationisimportant:IndatasetMwefoundcorrelationsbetweenthecodesforquestionsforinterpretations(Q_INT)andexperiencereports(EXP),andfordatasetEwefoundnegativecorrelationsbetweenthecodesforquestionsforinformation(Q_INF)andexperiencereports(EXP).
Inaddition,wefoundanegativecorrelationbetweencodesforsuggestionsbasedonexperiences(SUG_EXP)andquestionsformorePrillaMichaeletal.
726information(Q_INF).
Theseallsuggestthatquestionsforinterpretationarehelpfulforthearticulationofexperiences,whichthensupportscollaborativereection.
Forthesupportofonlinecollaborativereection,thismaymeanthatmechanismssuchasprompts(seeabove)may(pointusersto)askthesequestionsinordertostimulatecollaborativereection.
5.
2.
4.
LearningWedidnotobserveanycodescorrelatingwithindividualcodesthatdocumentlearning(S_LOOP,D_LOOP,andCHANGE),whichcanbeattributedtothefactthatinourdatasetsthesesinglelearningoutcomeswerenotrepresentedtoalargeextent(lownumberofexplicitstatementsonlearningasdescribedabove).
Asmentionedabove,thishasalsobeenobservedinotherstudies(deGrootetal.
2013;Prillaetal.
2015)anddoesnotmeanthatlearningdidnottakeplace(seethediscussionofthisintheexplanationofthecodesinsection3.
3),butthatitwasnotexplicitlydocumentedveryoften.
Infact,thiswasalsothecaseforourgroups:Asmentionedinsection4.
1ourparticipantstoldtousthattheylearnedfromdiscussionsintheplatform,andthisindicatesthatlearningwasoftennotdocumented.
Intheanalysisofallexplicitstatementsonlearning,wefoundthattherewasaconsiderableinuenceofthearticulationofexperiences(EXP)onlearningdocumentedinthetools,whichagainshowstheneedforarticula-tionworkincollaborativereection.
Onrstsight,thisseemstotthereectionmodelsdescribedinthispapernicely,asthearticulationofexpe-rienceisanintegralpartofreection.
Onasecondconsideration,however,thisleavesoutstepssuchascreatingsuggestionsbasedontheexperienceexchangeasdescribedbymostreectionmodels.
Anotherinterestingndingisthatmanyofthecorrelationsandcorre-spondingregressionmodelswefoundshowedhighesteffectsizesintheLearningclustercreatedduringtheanalysis–forbothdatasets.
Thisincludesrelationsbetweenquestionsforinterpretationandtheprovisionofexperiences,betweenlearningdocumentedandrelatingexperiencestoitandtherelationtosuggestionsfromexperiencetomoresuggestionsofthiskind,anditmeansthattheserelationswerestrongestwhenlearningoccurredinathread.
Thissuggeststhatlearningmayhavebeenmediatedbytheoccur-renceofthesecodecombinationsorviceversa.
Wedidnotndamediatingeffectintheanalysis,and,assuch,thisisanotheraspecttobetakenonboardinfurtherwork.
Wealsoobservedcorrelationsfromlearning(S_LOOP,D_LOOP,andCHANGE)tocodesforexperiencereports(EXP)orsuggestions(SUG),meaningthatbothexperiencereports(EXP)andsuggestions(SUG_KNOandSUG_EXP)followedtheexplicationoflearning.
Asmentionedearlier,thisindicatesthatfacil-itationmechanismsshouldnotstopfacilitatingreectivediscussiononcesomeonestatesthattheyhavelearned,asdiscussionsstillcontinue.
HowDoesCollaborativeReectionUnfoldinOnline.
.
.
7275.
3.
ResearchquestionsTheinsightsdiscussedaboveprovideagoodbasistoanswertheresearchquestionsguidingtheworkpresentedhere.
Below,werelatethemtotherespectivequestion.
RQ1:HowcanthenatureofonlinecollaborativereectionbedescribedWedidnotobserveclearorsingularreectionows,therelationswefounddeviatedfromwhatcouldhavebeenassumedbasedonexistingliterature,anddifferentrelationswerefoundinthedifferentdatasets.
Asdiscussedabove,thisshowsthattheremostlikelyaremultiplepathsofcollaborativereection,andthatthesearepotentiallyinuencedbydifferentfactors.
Thisopensupanewwaytolookatthenatureofcollaborativereectionasneithermodel-basednormessy,butconstitutedbymultiplepaths.
Forthesepaths,wefoundagoodamountofcausalrelationshipsbetweentypesofcontributionstocollaborativereection,whichmaymakeupelementsofthe"nature"welookedforinRQ1.
Thisincludesbothsequenceswellknownfromexistingmodelsandrelationshipsnotcommonlyfea-turedinexistingmodels(e.
g.
,theeffectofrelatingownexperiencestolearningdocumentedbyothers).
Ourndingsalsoshowtheimportanceofarticulationworkandreciprocityincollaborativereection.
Wefoundthataskingquestionsandsharingexperienceswerecorrelatedwithdocumentedlearningoutcomesinaconsiderableway,whichunderpinsthatarticulationofissuesiscrucialforreectingtogetherandneedstobesupport.
Wealsofoundreinforcement(agreeingwhenothersagreedindatasetE,agreeingonissueswhenexperiencesweresharedindatasetM)asanexampleforreciprocalinteractiontobecommonandofpotentialimportanceforcollaborativereection.
Theimportantroleofreciprocitywasalsofoundinmultiplesuggestionsco-occurring(datasetE)andstatementsonchangefollowingstatementsindicatinglearninghadhappeneddatasetM).
Alloftheseexamplescreateasituationinwhichtheparticipantsofacollaborativereectionprocessrelatetoeachotherandreecttogetherratherthansharingthoughtsandtakingtheseasinputforparallelindividualreections.
Additionally,wehavetotakeintoconsiderationthatinonlinecollaborativereec-tioncertainstepsmaybedonebytheusersofatoolsupportingcollaborativereectionoutsidethesystem,thatis,cognitivelywithoutbeingarticulatedinthetoolorinface-to-faceinteraction.
Thismayexplainwhythearticulationofexperienceshadadirectinuenceonlearning,whysuggestionsbasedonexperiencescameupwithoutexpe-riencesarticulatedinathreadbefore,andwhysomelearningoutcomeswerenotdocumented.
Ifthisisthecase,thenreectionsupportmayevenbenetfromthis,astherewouldbenoneedtoemphasizeorforcecertainstepsbutmechanismscouldrelyonusersoftenfulllingthesestepsontheirown.
Inanycase,wemustregardcollaborativereectioninonlinemediaasanonlineandofineprocess.
RQ1:(How)DoreectiveconversationsunfoldalongtheelementsmentionedincommonmodelsofreectionPrillaMichaeletal.
728WhiletheanswertoRQ1islargelybasedonourinsightsaboutthemultiplepathsthatcollaborativereectionmaytakeinpractice,theanswertoRQ2needstodescribethesepaths.
Themultiplicityofsequenceswefoundmeansthatfromourworkwecannot(andshouldnot)deriveasingledescriptionofhowreectionthreadsunfoldinonlinecommunities.
Comparingourndingstoexistingreectionmodels,thethreadsweanalyzedseemto"jump"certainstepsinthemodels,andotherstepsoccurwithoutthetriggersthatwouldbeexpected.
Ourndingsincludeinsightsonthepositiveinuenceofexperiencesexchange,questionsandtheprovisionofsuggestionsbasedonexperienceoncollaborativereection.
Therearealsonewinsightsthataddadditionalfacetsoncollaborativereection.
Oneoftheseisthatcollaborativereectionoftencontinuedinourgroupsafterlearninghadbeendocumented.
Thismayshowthat,asassumedinpreviouswork(Krogstieetal.
2013),collaborativereectiondoesnotterminatebutisaniterativeprocess.
Asanexampleofactivitiesmostlikelyperformedofine,wefoundthattheprovisionofexperiencescorrelatedwiththedocumentationoflearning,implicatingthatthestepofderivinglearninginsightswasdonecognitivelyorface-to-face.
AnimportantaspecttotakeawayfromtheanswerstoRQ1andRQ2isthatreectiveconversationsmaygodifferentwaysthansuggestedbymostreectionmodels,andthatreectionmaycreatevaluableoutcomesonallofthesedifferentways.
RQ1:WhicharetheelementsofreectionthatleadtosuccessfuloutcomesofreectionOurworksuggeststhatprovidingexperiencesinuencesreectionpositivelyandleadtodocumentedoutcomesandsuggestionsbasedonexperiences.
Wealsofoundthatsuggestionsbasedonexperienceswerefollowedbymoresuggestionsbasedonexperiences,andthatquestionsforinterpretationwerehelpfulforthecreationofoutcomes.
Theserelationsareinlinewiththeliterature,but(asdiscussedabove)representonlysomendingsamongstothers.
Besidesrelationsbetweencodesandcorrespondingutterancesthatweassumedtobepresent,wealsofoundsurprisingrelationsinthedata.
Amongthese,wefoundthatrelatingownexperiencesandknowledgetolearningdocumentedbyothersmayleadtounderstandingandadoptionofthislearningforoneself.
Thisisnotpromi-nentlyfeaturedinreectionmodelsandaddstothesemodels.
Inaddition,thisndingalsomaypointtoawayinwhichtheprovisionofknowledge,whichisusuallyreferredtoasnotbeinghelpfulforcollaborativereection,mayfosterlearningfromcollaborativereection.
Wealsoobservedthatagreementhasvariousinuencesonoutcomesofcollab-orativereection,asamongothersitisrelatedtosuggestionsbasedonknowledgeandsinglelooplearning.
Whilethismaysoundtrivialinitially,itpointstowardsaculturetobeestablishedinonlinecollaborativereectiontools,inwhichusersengagewithwhatothersshareandreassurethemthattheseexperienceshavebeenmadebyothersaswell.
Thisculturecannotbetakenforgrantedinmanyorganiza-tions,whichisalsosupportedbyresultswegatheredwhenweappliedthetoolHowDoesCollaborativeReectionUnfoldinOnline.
.
.
729resultingindatasetEinpractice(BlunkandPrilla2017b).
Intermsofsupport,facilitationmechanismsshouldpickthisupandencourageuserstoagreeanddisagreewitheachothertocontinuethediscussion.
OurobservationthatmanyofthecorrelationsandmodelswefoundshowedtheirhighesteffectsizesintheLearningclusterofthreadsinthedatasetssupportsthenotionthatthesecombinationsofcodesfollowingeachotherinthreadsarehelpfultoleadtooutcomesfromcollaborativereection.
RQ1:WhicharetheelementsthatdiminishreectionintheconversationsWefoundminorrelationsbetweencodesthatprovideinsightsonelementsorbehaviortoavoidincollaborativereection.
Amongthese,wefoundaweakrelationshipinwhichprovidingadviceledtosuggestionsbasedonknowledge,whicharenotfavorableincollaborativereection(asopposedtosuggestionsbasedonexperience).
Wealsofoundoccasionsinwhichadviceorsuggestionsbasedonknowledgeprovidedbyusersledto(singleloop)learning,whichisalsonotwhatisdesiredinreectionsupport.
Itshouldbenotedthatbotheffectsmaystillincludevaluableinsightsforusers,butthatthescopeofpureanalysiswasonfosteringlearningfromreection.
Moreover,asweobservedvariouscorrelationsoccurringintheNo-Learningclusters(e.
g.
multipleconsecutivequestionsformoreinformation(Q_INF)),wemayhypothesizeaneffectofthesekindsofquestionsonthreadsturningintoadirectionthatdoesnotleadtolearning.
Thisissupportedbythenegativecorrelationwefoundbetweenthistypeofquestionandtheprovisionofexperiences.
Facilitationsupportmaythereforetrytoencourageothertypesofquestionsasdescribedbelow.
5.
4.
ImpactsonmodelingreectionAlthoughreectionliteratureoftenimpliesasequenceofthingsthatneedtohappenbeforereectivelearningcantakeplace(seeSchn1983;Boudetal.
1985;Krogstieetal.
2013),ourdatadoesnotsupportthesemodelsfullybutshowsmuchmorevarietyandevenunexpectedrelations.
Forexample,fromthefrequencyanalysis,wefoundwecanseethatbothsolutionproposalsaswellaslearningoccurredrightfromthebeginning.
Ourresultsthereforesuggestthatthereisnotonesinglewaytodescribeonlinecollaborativereection(RQ1and2),andthattherewerebothcommonlyassumedandsurprisingelementsthatfostered(andhindered)reectioninthediscussionswelookedat(RQ3and4).
Thisaffectsthewaymodelsofcollaborativereectionshouldbebuiltandused.
Ratherthanshowingorimplicatingsequences,suchmodelsshouldinsteadfocusonthecollaborativeprocess.
Thismayincludehowpeoplerelatedtoeachother(interactivity,reciprocity)andhowonlinecollaborativereectionisnotonlyaprocessofindividualandcollaborativeactivityasdescribedinPrilla(Prilla2015),butalsoaprocessthathappensonlineandofineandthereforemissessometracesandtrajectoriesintheonlinemedium.
DealingwiththesegapsinthevisibilityandPrillaMichaeletal.
730availabilityofreectionaspectsforthegroupreectingtogethermeansallowingmoreexiblewaysforreectiontounfold(ratherthanprescribingpathsimplicatedbymodels)andtheneedtoemphasizethebenetcreatedbysharingtheseaspectswiththeotherparticipantsinordertoallowthemtorelatedtotheseaspects.
Ourndingsalsosuggestthatweneedtocarefullyuseexistingmodelswhenanalyzinganddesigningforcollaborativereection.
Usingthesemodelstoexplainonlinecollaborativemodelsmayautomaticallyresultinanincompleteviewofwhathappensinreectiveconversationsandthequestionariseswhethermodelscancapturecollaborativereectionafterall,giventhediversitywefound.
Thismaybethetruemeaningofreectionbeing'tamedanddomesticatedattheriskofdestroyingwhatitcanoffer'asstatedbyCresseyetal.
(2006,p.
23).
Ontheotherhand,wemustnotstopatthenotionthatreectionismessy,aswehaveshownthattherearemultiplepathsthatitunfoldsalong.
Inanycase,ourworkpointstowardstheneedtoatleastpartiallyreconsiderthesemodels,especiallylinkingournewndingstothem.
Furtherworkinthisdirection,asstatedabovemanytimes,needsfurtherinvestigationofourndingsandadditionaldatatoexamine.
5.
5.
Designingfor(collaborative)reection:implicationsforfacilitationsupportInouranalysiswegainedseveralinsightsintohowcollaborativereectioninonlinediscussionunfoldsandwhichfactorsmightinuenceothers.
Oneimplicationfromourworkisthatfacilitationmechanisms,whichhavebeenpresentedaskeytothesupportforcollaborativereectionabove,maynothavetostrictlyadheretospecicstepsorprerequisitesinordertosupportreection,butprovidefreedomforcollab-orativereectiontounfoldalongdifferentpathsanddirectlyaskuserstoprovidecertaincontributions.
Basedonouranalysiswederivedseveralsuggestionstodealwiththevarietyofwayswefoundforcollaborativereection:&Facilitationmechanismsshouldsupportusersinarticulatingandsharingtheirexperiences,inrelatingtheirstatementstoeachother(reciprocity),andinprovidingsolutionproposalsbasedonexperiencefromthebeginningofathreadon.
Bothdescribepathsthatledtooutcomesinthethreadsweanalyzed.
Onewayofprovidingthissupportcouldbeinpromptinguserstoarticulatecorrespondingcontributions.
Ourinitialworkonpromptsforreectionsupportsthis(BlunkandPrilla2017a;Renneretal.
2016),butfurtherworkisneededtobuildandevaluatethissupport.
&Facilitationmechanismsshouldencourageuserstocontinuereectivediscussionevenafterlearninghasbeendocumentedbyreferringtotheirexperiencesandknowledgetothedocumentedlearningoutcomes.
Fortheimplementationofsupportthismeanstwothings:First,mechanismsneedtohelpuserstomakedocumentedlearningvisibletoothers,asthismayhelpotherstobenetfromit.
Second,promotingmechanismsshouldencourageuserstoarticulaterelatedexperiencesinordertohelpthemtolearnaswell.
However,supportforcollaborativereectionalsoneedstotakeintoaccountsitsnatureasanonlineandHowDoesCollaborativeReectionUnfoldinOnline.
.
.
731ofineprocess,andthereforeweshouldnotforceuserstoexplicateeverythingbutallowthemtocarryoutphasesinthewaystheycanreectbest.
&Facilitationmechanismsshouldfosteracultureinwhichusersengagewithotherusers'contributionsandexplicitlyagreeordisagreewithitintheircontributions.
Bothwaysofreferringtoothercontributionswerefoundtobeusefulinouranalysis.
Thiscouldbedonebyencouragingandshowcasingrespectiveopennessandengagement.
Inaddition,promptsmaymakeusersawareofthepositiveaspectsofengagingwithothercontent.
&Facilitationmechanismsshouldencourageandhelpuserstoaskquestionsforinterpretationofcontentsharedwiththem,asinouranalysisthisshowedtosupportreection.
Onewaytogentlydirectonlinediscussionsindirectionsthatfostercollaborativereectionistoprovideuserswithblueprintsforsuchquestions,whichtheycanadoptandintegrateintotheircontributions.
Wehaveimplementedaprototypeforthis,andatthetimeofwritingthispaper,weareevaluatingit(seeBlunkandPrilla2017bforexamplesandveryearlyresults).
&Facilitationmechanismsshouldadapttothespeciccontextofthecollaborativereectionandtotheuser(s).
Enablinguserstotakemultiplepathsincollaborativereectionneedscontext-dependentmechanismsthatprovidesupportfortherespectivereectionpathathreadfollowsorshouldfollowaccordingtoitscontextorusers.
Itmayalsoaffordthepersonalizationoffacilitation,thatis,providingtherightkindofsupportforacertainuser.
However,whetherandhowthesuccessofcertainfacilitationsupportcanberelatedtothesituationorthepersonreceivingthesupportissubjecttofurtherwork.
Ourworksuggeststhatthisisapathworthwhilefollowing.
Wealsonoticedthatthedatasetsdifferedintheinuenceswefoundoncollab-orativereection.
Weattributedthistothedifferenceinthecontextsthedatastemsfrom.
Forthedesignofreectionsupport,thismeansthatweneedtounderstandthecontext(e.
g.
,smallvs.
largegroups,shortorlong-termexposureetc.
),whatitmeansfortheowofreection,andtailorsupporttoit.
Followinguponourdiscussionabove,insmallergroupswemayhavetopromptforactivitythatincludessensemakingsuchastheprovisionofadditionalexperiences,whileinlargergroupswemaydirectlyaskforsolutionproposals.
However,whileourworkpointsatdifferencesinthecontext,itcanonlyprovideinitialinsightsandfurtherworkisneededtoexplorethisinuence.
Fortheimplementationofcontext-dependentsupport,thereisaneedtogainanunderstandingofwhatishappeninginacollaborativereectionthread.
Themanualcontentcodingapproachthatweemployedtoanalyzethedata,whilebeingwell-suitedforthisanalysis,doesnotworkforthat.
Instead,thereisaneedforon-the-spotdetectionofthesituationinthread.
NumeroustoolsforautomatedcontentanalysissuchasLIWC(TausczikandPennebaker2010)andEMPATH(Fastetal.
2016)exist.
However,asstatedabove,thesetoolsdonotincludeelementsneededtoanalyzecollaborativereection.
AddingtheseelementsbasedontheworkpresentedPrillaMichaeletal.
732hereandotherworkaswellasusingtheirexistingelementstoanalyze,forexample,thetopicsreecteduponcouldenableustoautomaticallyassessandcategorizethecontent.
Inthiscase,facilitationfeaturescouldbeselectedbasedonwhatmaybemosthelpfulinathread.
This,however,needsadditionalwork,asnoneoftheexistingtoolscontainsthecategoriesincludedinthecodingschemeweused.
5.
6.
LimitationsOurworkbringsforwardinterestinginsightsonhowcollaborativereectionunfoldsinonlinecommunities,andhowtopotentiallysupportit.
However,weawareofthefactthatourndingscannotbegeneralized.
Wediscusslimitationsofourworkbelow.
Ourndingsarebasedonsequencesanalysesandcorrelationsbackedupbycorrespondingmodelsfromlinearregressionanalysis.
Whilethisissufcienttodrawthetentativeconclusionswedrawinthepaper(e.
g.
,multiplepathsofreectionthatarepotentiallyinuencedbycontextualfactors),wedidnotndparticularpatterns,andtheexplanatorypowerofsomemodelsislow.
Wementionedthisexplicitlyinourresults,andwemarkedthoserelationsbetweencontributionstocollaborativereectionthatneedfurtherinvestigation.
However,despitethislimitation,ourworkclearlyshowsthepluralityofpathsthatcollaborativereectionmaytake,anditindicatesthatwemayneedtosupportsomeofthesedifferentpathsratherthanfollowingcertainmodels.
Inaddition,giventheamountofdata,wewerealmostguaranteedtoobserveseveralcorrelations.
Thus,wefocusedoncorrelationsappearingmultipletimes(thatis,intheoveralldatasetsandintheclusterswecreated)aswellasoncorrelationsthatwefoundacceptableregressionmodelsfor.
Despitethis,weemphasizethatfurtherworkisneededtoapproveandstrengthenourndings,whilewealsoemphasizethatthesendingsprovidenewinsightsontheirown.
Wealsoappliedthecontentanalysistocontentwritteninalanguagetheauthorsdonotspeak.
ThecodingschemebyPrillaetal.
(2015)usedwasdevelopedandevaluatedfortheanalysisofGermanandEnglishtexts,andthereforeitisnotguaranteedthatitpicksupalllanguageandculturerelatedsubtletiesofthislanguage.
Inaddition,codingneededtobedonebydifferentcodersforthedatasetsduetothislanguagebarrier.
WeaccountedfortheseissuesbytrainingthecodersfordatasetEwiththecodingappliedtodatasetMbytheresearchers,makingsuretherewasahigherinterraterreliabilitybetweenthecodersandtheresearchersinordertomakesurethatthecoders'understandingofthecodingschemewasthesamethanthatoftheresearchers.
Thiswaywecouldmakesurethatbothgroupsofcodershadacomparableunderstandingofthecodingscheme.
6.
ConclusionThispaperprovidesananalysisofreectiveconversationsfromtwoonlinetoolsgatheredindifferentworkplacesettings.
Theanalysisisbasedonamanualcontentcodingschemedevelopedfortheinvestigationofcollaborativereection.
WeHowDoesCollaborativeReectionUnfoldinOnline.
.
.
733employeddifferentmethodsofanalysistoprovideanswerstothefourresearchquestionsguidingourwork.
Fromthisanalysis,wederivedanumberofinsightsintothestructuresofthosediscussions,amongwhichsomesupportexistingknowl-edgeoncollaborativereection,whilemanyothersquestionthisknowledgeandaddtoit.
Mostimportantly,wefoundthatcollaborativereectionunfoldsalongmultiplepaths,whicharelikelytobeinuencedbythecontextreectionisconductedin.
Whilethismayseemintuitiveoreventrivial,thereishardlyanyworkthatpointsintothisdirectionandnoworkthatidentiesanddescribesthepaths,whichisdoneintheworkpresentedhere.
Inaddition,ourresultsindicatethatexperiencesandsugges-tionsbasedonexperiencesleadtooutcomesofreection,andthatengagingwithcontributionsofothersandaskingtherightquestionsfostersthisprocessaswell.
Whilethiscouldhavebeenassumedfromtheliterature,wealsofoundthatreectionmaybesuccessfulonmanymoredifferentpathsthandiscussedintheliterature.
Twoexamplesforthisarethatprovidingexperiencesafterlearninghasbeendocumentedmaybeimportantforlearningfromcollaborativereection,andthatsuggestionsbasedonownexperiencescanbeprovidedwithoutprerequisitesorothercontentpresent.
Basedontheseinsightsweprovideseveralsuggestionsonhowreectioncouldbesupportedbyfacilitationmechanisms,includingadiscussionhowtoapplythesemechanismsandfurtherworkneededforthis.
Whileweareawareofthelimitationsofourwork,ourresultsprovideinterestingandnovelinsightsintothecourseofcollaborativereection,andtheinsightsfromouranalysiscanbeusedtoinspirethedesignofmeanstofacilitateandsupportreection.
AcknowledgmentsOpenAccessfundingprovidedbyProjektDEAL.
ThisworkwaspartoftheEmployID(http://employid.
eu)projecton"Scalable&cost-effectivefacilitationofprofessionalidentitytransformationinpublicemploymentservices"supportedbytheECinFP7(projectno.
619619).
Wethankallcolleaguesandassociatedpartnersfortheircooperationandourfruitfuldiscussions.
SpecialthankstoourstudentresearchersNikaandNea.
OpenAccessThisarticleislicensedunderaCreativeCommonsAttribution4.
0InternationalLicense,whichpermitsuse,sharing,adaptation,distributionandreproductioninanymediumorformat,aslongasyougiveappropriatecredittotheoriginalauthor(s)andthesource,providealinktotheCreativeCommonslicence,andindicateifchangesweremade.
Theimagesorotherthirdpartymaterialinthisarticleareincludedinthearticle'sCreativeCommonslicence,unlessindicatedotherwiseinacreditlinetothematerial.
Ifmaterialisnotincludedinthearticle'sCreativeCommonslicenceandyourintendeduseisnotpermittedbystatutoryregulationorexceedsthepermitteduse,youwillneedtoobtainpermissiondirectlyfromthecopyrightholder.
Toviewacopyofthislicence,visithttp://creativecommons.
org/licenses/by/4.
0/.
PrillaMichaeletal.
7341.
Appendix1.
1.
AppendixA:ContentcodingschemeTable7.
Contentcodingschemeusedintheanalysis(adaptedfrom(Prillaetal.
2015)).
Toimprovereadabilitywesubstitutedtheoriginalcodenumberswithlabelsdepictingthemeaningofthecodes.
CodeDescription(adaptedfrom(Prillaetal.
2015))EMO_OWNMentioningownemotionsinanexperience(e.
g.
'Wasnotfunman'or'thismademeangry')withanexplicitemotion-keywordbeingpresent(e.
g.
'happy','sad','frustrated','angry').
EMO_OTHMentioningemotionsofothersinanexperience(e.
g.
'[resident]saidheisunhappylivinghere')withanexplicitemotion-keywordbeingpresent(e.
g.
'happy','sad','frustrated','angry').
INTInterpretationorjusticationofactionsandsituations(e.
g.
'AsfarasIamawareIhaddonenothingtodeservethis').
Thiscodeisassignedwhenthethreadauthoraddsinterpretationorjustication.
Thisincludesassessmentsofthesituation(e.
g.
,explanationswhythesituationisproblematicorrelevantforworkgoingbeyondonlydescribingtheproblem),aswellashypothesesforproblems/success.
EXPLinkinganexperiencetootherexperiences(e.
g.
'Imadeasimilarexperience',or'Xtoldmehewasthroughthisbefore…').
Explicitreferencetopastexperienceneeded,otherwiseadifferentcodeisused(e.
g.
KNO).
KNOLinkinganexperiencetoknowledge,rulesorvaluesnotexplicitlylinkedtoaparticularexperience(e.
g.
'Iwasreadinganarticle(guardiannewspaper)ondo'sanddon'tsinaninterview.
Oneofadon'twas,nottokeeplookingintheintervieweeseyesbutlookattheirface').
DISAGRRespondingtointerpretationoftheactionbychallengingexistinginterpretation(s)/suggestionsoraddingperspectives(e.
g.
'Hmmm.
Isthisreallydifferentfrom[…]').
AGRRespondingtointerpretationoftheactionbysupportinginterpretation(s)andsuggestions(e.
g.
'Agreed!
').
Thisincludesalsoauthorsrepeatingprevioussuggestions.
ADVGivingadvicewithoutareasonorreferencetoexperiences(e.
g.
'Neveracceptblameforanother'smistake').
SUGaGivingpossiblesolutionswithareasongiven(seeexamplesforcodesSUG_EXP,SUG_KNO).
SUG_EXPbGivingpossiblesolutionswithareasongivenbasedonexperience(e.
g.
'frommyexperiencealistofFAQsisuseful').
Whenthiscodeisassigned,codeEXPisassignedtotheexperiencepartofthesolution,astheauthoralsomakesareferencetoherpersonalexperiences.
SUG_KNO2Givingpossiblesolutionswithareasongivenbasedonknowledge(e.
g.
'ItisgoodtodoX,becauseithelpsto…').
Whenthiscodeisassigned,codeKNOisassignedtotheknowledgepartofthesolutionproposal,astheauthoralsomakesareferencetoherknowledge.
(continuedonnextpage)HowDoesCollaborativeReectionUnfoldinOnline.
.
.
7351.
2.
AppendixB:DatafromtheanalysisTable7.
(continued)CodeDescription(adaptedfrom(Prillaetal.
2015))S_LOOPInsightsfromreectionassingle-looplearning:Differentorbetterunderstandingofexperiences.
Expressedbyreportinginsights(e.
g.
'IrealisedthatIshouldn'thavehavebeensoworriedaboutthis[…]'or'ItisgoodtoknowthatIpersonallyhaven'tdonesomethingwrong').
Thiscodeisonlyassignedifthelearningresultedfromthecurrentreectionprocesswithinthediscussion,andstatementsaboutpriorlearningarenotcoded.
D_LOOPInsightsfromreectionasdouble-looplearning:Generalizingfromexperiences.
Expressedbypatternsorrootsofaproblem(e.
g.
'ThebestwayIfoundtodealwiththisis[…]').
Thiscodeisonlyassignedifthelearningresultedfromthecurrentreectionprocesswithinthediscussion,andstatementsaboutpriorlearningarenotcoded.
CHANGEDrawingconclusionsandimplicationsfromreectionbysuggestingtoapplynewpracticeorreportingonchangesdoneorplanned,likehowtoimplementachange(e.
g.
'Willdenitelytryanddo[…]inthefuture').
Q_INFQuestionsforinformationontheexperience,ifnotallinformationisgivenoradditionalinformationissupposedtobeavailable(e.
g.
'whatdoyoumeanby[…]'or'whathappenedafterwards').
Q_INTQuestionstriggeringdiscussions,askingforaninterpretationofasituation,foropinionsorproposals(e.
g.
'whatdopeoplethinkabout[…]').
Thiscodeisalsoassignedforquestionsaimedtostartdiscussionsorkeepthemrunning(e.
g.
'whatdopeoplethinkabout[…]'or'howcanwesolvethisproblem')aCodeSUGwasusedindatasetMonlybCodesSUG_EXPandSUG_KNOwereusedindatasetEonlyTable8.
Overviewofallreportedndingsacrossbothdatasets.
Eachrowshowsoneapossibleinuenceofthecodeinalloftheortheimmediatepredecessorpostandthesubsequentposttogetherwiththehighestobservedcorrelationandtheresultoftheregressionanalysisforthatspecicpair.
Theregressionanalysishasbeenperformedontheclustercontainingthehighestcorrelation.
PredecessorCurrentcodeCorrelationRegressionDataSetM–AllandImmediatePredecessorsCHANGEEXPr=0.
554,p<0.
001r2=0.
307,F=33.
615,p<0.
001Q_INTEXPr=0.
516,p=0.
001r2=0.
266,F=12.
697,p=0.
001EXPAGRr=0.
435,p=0.
002r2=0.
189,F=10.
978,p=0.
002KNOAGRr=0.
428,p=0.
002r2=0.
183,F=10.
551,p=0.
002D_LOOPAGRr=0.
557,p<0.
001r2=0.
310,F=32.
367,p<0.
001S_LOOPSUGr=0.
561,p<0.
001r2=0.
315,F=16.
081,p<0.
001Q_INFQ_INFr=0.
703,p<0.
001r2=0.
083,F=6.
910,p=0.
01DataSetE–AllandImmediatePredecessors(continuedonnextpage)PrillaMichaeletal.
736ReferencesArgyris,Chris;andDonaldA.
Schn(1978).
Organizationallearning:ATheoryofActionPerspective.
Reading,Mass.
:Addison-WesleyPub.
Co.
Barrows,HowardS.
(1986).
Ataxonomyofproblem-basedlearningmethods.
MedicalEducation,vol.
20,no.
6,November1986,pp.
481–486.
Baumer,EricP.
S.
(2015).
ReectiveInformatics:ConceptualDimensionsforDesigningTechnologiesofReection.
CHI'15.
Proceedingsofthe33rdAnnualACMConferenceonHumanFactorsinComputingSystems,Seoul,RepublicofKorea,18-23April2015.
NewYork,NY,USA:AssociationforComputingMachinery,pp.
585–594.
Bjrn,Pernille;andNinaBoulus(2011).
Dissentinginreectiveconversations:Criticalcomponentsofdoingactionresearch.
ActionResearch,vol.
9,no.
3,September2011,pp.
282–302.
Blunk,Oliver;andMichaelPrilla(2017a).
PromptingtoSupportReection:AWorkplaceStudy.
In.
Lavoué,H.
Drachsler,K.
Verbert,J.
Broisin,&M.
Pérez-Sanagustín(eds.
):ECTEL'17.
DataDrivenApproachesinDigitalEducation,Tallinn,Estonia,12-15September2017.
Cham:Springer,pp.
367–372.
Blunk,Oliver;andMichaelPrilla(2017b).
SupportingCommunitiesofPracticeinPublicAdministrations:FactorsInuencingAdoptionandReadiness.
C&T'17.
Proceedingsofthe8thInternationalConferenceonCommunitiesandTechnologies,Troyes,France,26-30June2017.
NewYork,NY,USA:ACM,pp.
36–45.
Boud,David;RosemaryKeogh;andDavidWalker(1985).
Promotingreectioninlearning:Amodel.
InD.
Boud,R.
Keogh,&D.
Walker(eds.
):Reection:TurningExperienceIntoLearning.
London,NewYork:RoutledgeFalmer,Taylor&FrancisGroup,pp.
18–40.
Clark,Herbert;andSusanE.
Brennan(1991).
GroundinginCommunication.
InL.
B.
Resnick,J.
M.
Levine,&S.
D.
Teasley(eds.
):PerspectivesonSociallySharedCognition.
Washington,DC:AmericanPsychologicalAssociation,pp.
127–149.
Convertino,Gregorio;ThomasP.
Moran;andBartonA.
Smith(2007).
StudyingactivitypatternsinCSCW.
CHI'07.
ExtendedAbstractsonHumanFactorsinComputingSystems,SanJose,CA,USA,28April-3May2007.
NewYork,NY,USA:AssociationforComputingMachinery,pp.
2339–2344.
Table8.
(continued)PredecessorCurrentcodeCorrelationRegressionSUG_EXPSUG_EXPr=0.
326,p=0.
001r2=0.
106,F=12.
716,p=0.
001ADVSUG_KNOr=0.
552,p<0.
001r2=0.
103,F=6.
740,p=0.
012SUG_EXP-Q_INTr=0.
477,p<0.
001r2=0.
227,F=34.
125,p<0.
001DataSetM–OnlyImmediatePredecessorsADVEXPr=0.
561,p<0.
001r2=0.
315,F=33.
081,p<0.
001D_LOOPEXPr=0.
441,p<0.
001r2=0.
194,F=17.
382,p<0.
001D_LOOPKNOr=0.
301,p=0.
01r2=0.
091,F=6.
985,p=0.
010D_LOOPCHANGEr=0.
388,p=0.
001r2=0.
151,F=12.
781,p=0.
001DataSetE–OnlyImmediatePredecessorsAGRAGRr=0.
381,p=0.
002r2=0.
145,F=9.
998,p=0.
002AGRS_LOOPr=0.
393,p=0.
002r2=0.
154,F=10.
747,p=0.
002S_LOOPD_LOOPr=0.
338,p<0.
001r2=0.
114,F=14.
981,p<0.
001DataSetE–AllPredecessorsQ_INF-EXPr=0.
302,p=0.
018r2=0.
091,F=5.
916,p=0.
018HowDoesCollaborativeReectionUnfoldinOnline.
.
.
737Cressey,Peter;DavidBoud;andPeterDocherty(2006).
Theemergenceofproductivereection.
InD.
Boud,P.
Cressey,&P.
Docherty(eds.
):Productivereectionatwork:Learningforchangingorganizations.
London:Routledge,pp.
11–26.
Daudelin,MarilynWood(1996).
Learningfromexperiencethroughreection.
OrganizationalDynamics,vol.
24,no.
3,Winter1996,pp.
36–48.
Davis,ElizabethA.
(2000).
Scaffoldingstudents'knowledgeintegration:PromptsforreectioninKIE.
InternationalJournalofScienceEducation,vol.
22,no.
8,August2000,pp.
819–837.
deGroot,Esther;MaaikeD.
Endedijk;A.
DebbieC.
Jaarsma;P.
Robert-JanSimons;andPetervanBeukelen(2013).
Criticallyreectivedialoguesinlearningcommunitiesofprofessionals.
StudiesinContinuingEducation,vol.
36,no.
1,May2013,pp.
15–37.
DeWever,Bram;TammySchellens;MartinValcke;andHildeVanKeer(2006).
Contentanalysisschemestoanalyzetranscriptsofonlineasynchronousdiscussiongroups:Areview.
Computers&Education,vol.
46,no.
1,January2006,pp.
6–28.
Dewey,John(1933).
Howwethink:Arestatementoftherelationofreectivethinkingintheeducativeprocess.
Chicago:HenryRegnery.
Dillenbourg,Pierre;SannaJrvel;andFrankFischer(2009).
TheEvolutionofResearchonComputer-SupportedCollaborativeLearning.
InD.
N.
Balacheff,D.
S.
Ludvigsen,D.
T.
deJong,D.
A.
Lazonder,&D.
S.
Barnes(eds.
):Technology-EnhancedLearning.
Dordrecht:SpringerNetherlands,pp.
3–19.
Dyke,Martin(2006).
Theroleofthe'Other'inreection,knowledgeformationandactioninalatemodernity.
InternationalJournalofLifelongEducation,vol.
25,no.
2,March2006,pp.
105–123.
Eraut,Michael(2004).
Informallearningintheworkplace.
StudiesinContinuingEducation,vol.
26,no.
2,July2004,pp.
247–273.
Fast,Ethan;BinbinChen;andMichaelS.
Bernstein(2016).
Empath:UnderstandingTopicSignalsinLarge-ScaleText.
CHI'16.
Proceedingsofthe2016CHIConferenceonHumanFactorsinComputingSystems,SanJose,CA,USA,7-12Mai2016.
NewYork,NY,USA:AssociationforComputingMachinery,pp.
4647–4657.
Fessl,Angela;OliverBlunk;MichaelPrilla;andViktoriaPammer(2017).
TheknownUniverseofReectionGuidance:ALiteratureReview.
InternationalJournalofTechnologyEnhancedLearning,vol.
9,no.
2–3,January2017,pp.
103–125.
Fleck,Rowanne;andGeraldineFitzpatrick(2009).
Teachers'andtutors'socialreectionaroundSenseCamimages.
InternationalJournalofHuman-ComputerStudies,vol.
67,no.
12,December2009,pp.
1024–1036.
Fleck,Rowanne;andGeraldineFitzpatrick(2010).
Reectingonreection:Framingadesignlandscape.
OZCHI'10.
Proceedingsofthe22ndConferenceoftheComputer-HumanInteractionSpecialInterestGroupofAustraliaonComputer-HumanInteraction,Brisbane,Australia,22-26November2010.
NewYork,NY,USA:AssociationforComputingMachinery,pp.
216–223.
Gee,JamesPaul(2004).
DiscourseAnalysis:WhatMakesItCriticalInR.
Rogers(ed.
):AnIntroductiontoCriticalDiscourseAnalysisinEducation.
NewYork,NY,USA:Routledge,pp.
23–46.
Gray,Paul;BobJohansen;JayNunamaker;JeffRodman;andGeraldR.
Wagner(2011).
GDSSPast,Present,andFuture.
InD.
Schuff,D.
Paradice,F.
Burstein,D.
J.
Power,&R.
Sharda(eds.
):DecisionSupport:AnExaminationoftheDSSDiscipline.
NewYork,NY:Springer,pp.
1–24.
Hatton,Neville;andDavidSmith(1995).
Reectioninteachereducation:Towardsdenitionandimplementation.
TeachingandTeacherEducation,vol.
11,no.
1,January1995,pp.
33–49.
Hayes,AndrewF.
;andKlausKrippendorff(2007).
AnsweringtheCallforaStandardReliabilityMeasureforCodingData.
CommunicationMethodsandMeasures,vol.
1,no.
1,April2007,pp.
77–89.
Hmelo-Silver,CindyE.
(2004).
Problem-basedlearning:WhatandhowdostudentslearnEducationalPsychologyReview,vol.
16,no.
3,September2004,pp.
235–266.
PrillaMichaeletal.
738Hoyrup,Steen(2004).
Reectionasacoreprocessinorganisationallearning.
JournalofWorkplaceLearning,vol.
16,no.
8,December2004,pp.
442–454.
Introne,JoshuaE.
;andMarcusDrescher(2013).
Analyzingtheowofknowledgeincomputermediatedteams.
CSCW'13.
Proceedingsofthe2013conferenceonComputersupportedcooperativework,SanAntonio,Texas,USA,23-27February2013.
NewYork,NY,USA:AssociationforComputingMachinery,pp.
341–356.
Isaacs,Ellen;ArtieKonrad;AlanWalendowski;ThomasLennig;VictoriaHollis;andSteveWhittaker(2013).
EchoesfromthePast:HowTechnologyMediatedReectionImprovesWell-being.
CHI'13.
ProceedingsoftheSIGCHIConferenceonHumanFactorsinComputingSystems,Paris,France,27April-2May2013.
NewYork,NY,USA:ACM,pp.
1071–1080.
Jrvinen,Annikki;andEsaPoikela(2001).
Modellingreectiveandcontextuallearningatwork.
JournalofWorkplaceLearning,vol.
13,no.
7–8,October2001,pp.
282–289.
Johri,Aditya;andSeungwonYang(2017).
ScaffoldedHelpforLearning:HowExpertsCollaborativelySupportNewcomerParticipationinOnlineCommunities.
C&T'17.
Proceedingsofthe8thInternationalConferenceonCommunitiesandTechnologies,Troyes,France,26-30June2017.
NewYork,NY,USA:ACM,pp.
149–158.
Kim,Dongsik;andSeungheeLee(2002).
DesigningCollaborativeReectionSupportingToolsine-Project-BasedLearningEnvironments.
JournalofInteractiveLearningResearch,vol.
13,no.
4,2002,pp.
375–392.
Kolb,DavidA.
(1984).
ExperientialLearning:ExperienceastheSourceofLearningandDevelopment.
EnglewoodCliffs,NewJersey:Prentice-Hall.
Krogstie,Birgit;MichaelPrilla;andViktoriaPammer(2013).
UnderstandingandSupportingReectiveLearningProcessesintheWorkplace:TheCSRLModel.
InD.
Hernández-Leo,T.
Ley,R.
Klamma,&A.
Harrer(eds.
):ECTEL'13,ScalingupLearningforSustainedImpact,Paphos,Cyprus,Greece,17-21September2013.
Berlin,Heidelberg:SpringerBerlinHeidelberg,pp.
151–164.
Lin,Xiaodong;andJamesD.
Lehman(1999).
Supportinglearningofvariablecontrolinacomputer-basedbiologyenvironment:Effectsofpromptingcollegestudentstoreectontheirownthinking.
JournalofResearchinScienceTeaching,vol.
36,no.
7,August1999,pp.
837–858.
Marcu,Gabriela;AnindK.
Dey;andSaraKiesler(2014).
Designingforcollaborativereection.
PervasiveHealth'14.
Proceedingsofthe8thInternationalConferenceonPervasiveComputingTechnologiesforHealthcare,Oldenburg,Germany,20-23May2014.
Brussels,Belgium:ICST(InstituteforComputerSciences,Social-InformaticsandTelecommunicationsEngineering),pp.
9–16.
Mercer,Neil;andRupertWegerif(1999).
Is"exploratorytalk"productivetalkInK.
Littleton&P.
Light(eds.
):Learningwithcomputers:Analysingproductiveinteraction.
London,NewYork:Routledge,pp.
79–101.
Moon,JenniferA.
(1999).
Reectioninlearningandprofessionaldevelopment:Theoryandpractice.
London:KoganPage.
Newman,DavidR.
;BrianWebb;andCliveCochrane(1995).
Acontentanalysismethodtomeasurecriticalthinkinginface-to-faceandcomputersupportedgrouplearning.
InterpersonalComputingandTechnology,vol.
3,no.
2,April1995,pp.
56–77.
Pea,RoyD.
(2004).
TheSocialandTechnologicalDimensionsofScaffoldingandRelatedTheoreticalConceptsforLearning,Education,andHumanActivity.
JournaloftheLearningSciences,vol.
13,no.
3,July2004,pp.
423–451.
Pei,Jian;JiaweiHan;B.
Mortazavi-Asl;JianyongWang;H.
Pinto;QimingChen;U.
Dayal;andMei-ChunHsu(2004).
Miningsequentialpatternsbypattern-growth:ThePrexSpanapproach.
IEEETransactionsonKnowledgeandDataEngineering,vol.
16,no.
11,November2004,pp.
1424–1440.
HowDoesCollaborativeReectionUnfoldinOnline.
.
.
739Prilla,Michael(2015).
SupportingCollaborativeReectionatWork:ASocio-TechnicalAnalysis.
AISTransactionsonHuman-ComputerInteraction,vol.
7,no.
1,January2015,pp.
1–17.
Prilla,Michael;andBettinaRenner(2014).
SupportingCollaborativeReectionatWork:AComparativeCaseAnalysis.
GROUP'14.
Proceedingsofthe18thInternationalConferenceonSupportingGroupWork,SanibelIsland,Florida,USA,9-12November2014.
NewYork,NY,USA:AssociationforComputingMachinery,pp.
182–193.
Prilla,Michael;MartinDegeling;andThomasHerrmann(2012).
CollaborativeReectionatWork:SupportingInformalLearningataHealthcareWorkplace.
GROUP'12.
Proceedingsofthe17thACMinternationalconferenceonSupportinggroupwork,SanibelIsland,Florida,USA,27-31October2012.
NewYork,NY,USA:AssociationforComputingMachinery,pp.
55–64.
Prilla,Michael;ViktoriaPammer;andBirgitKrogstie(2013).
FosteringCollaborativeRedesignofWorkPractice:ChallengesforToolsSupportingReectionatWork.
InO.
W.
Bertelsen,L.
Ciol,M.
A.
Grasso,&G.
A.
Papadopoulos(eds.
):ECSCW2013:Proceedingsofthe13thEuropeanConferenceonComputerSupportedCooperativeWork,Paphos,Cyprus,Greece,21-25September2013.
London:Springer,pp.
249–268.
Prilla,Michael;AlexanderNolte;OliverBlunk;DennisLiedtke;andBettinaRenner(2015).
AnalyzingCollaborativeReectionSupport:AContentAnalysisApproach.
InN.
Boulus-Rdje,G.
Ellingsen,T.
Bratteteig,M.
Aanestad,&P.
Bjrn(eds.
):ECSCW'15.
Proceedingsofthe14thEuropeanConferenceonComputerSupportedCooperativeWork,Oslo,Norway,19-23September2015.
Cham:SpringerInternationalPublishing,pp.
123–142.
Prinz,Wolfgang;andBaberZaman(2005).
Proactivesupportfortheorganizationofsharedworkspacesusingactivitypatternsandcontentanalysis.
GROUP'05.
Proceedingsofthe2005internationalACMSIGGROUPconferenceonsupportinggroupwork,SanibelIsland,Florida,USA,6-9November2005.
SanibelIsland,Florida,USA:AssociationforComputingMachinery,pp.
246–255.
Raelin,JosephA.
(2002).
"IDon'tHaveTimetoThink!
"(vs.
TheArtofReectivePractice).
Reections,vol.
4,no.
1,Fall2002,pp.
66–79.
Renner,Bettina;MichaelPrilla;UlrikeCress;andJoachimKimmerle(2016).
EffectsofPromptinginReectiveLearningTools:FindingsfromExperimentalField,Lab,andOnlineStudies.
FrontiersinPsychology,vol.
7,no.
820,May2016,pp.
1–9.
Robertson,Toni(2002).
ThePublicAvailabilityofActionsandArtefacts.
ComputerSupportedCooperativeWork(CSCW),vol.
11,no.
3,September2002,pp.
299–316.
Roschelle,Jeremy;andStephanieD.
Teasley(1995).
TheConstructionofSharedKnowledgeinCollaborativeProblemSolving.
InC.
O'Malley(ed.
):ProceedingsfromtheNATOAdvancedResearchWorkshoponComputerSupportedCollaborativeLearning,AcquafreddadiMaratea,Italy,24-28September1989.
SpringerBerlinHeidelberg,pp.
69–97.
Schmidt,Kjeld;andLiamBannon(1992).
TakingCSCWSeriously:SupportingArticulationWork.
ComputerSupportedCooperativeWork(CSCW),vol.
1,no.
1–2,March1992,pp.
7–40.
Schn,DonaldA.
(1983).
TheReectivePractitioner:HowProfessionalsThinkinAction.
NewYork:BasicBooks.
Scott,SusanneG.
(2010).
EnhancingReectionSkillsThroughLearningPortfolios:AnEmpiricalTest.
JournalofManagementEducation,vol.
34,no.
3,June2010,pp.
430–457.
Slovák,Petr;ChristopherFrauenberger;andGeraldineFitzpatrick(2017).
ReectivePracticum:AFrameworkofSensitisingConceptstoDesignforTransformativeReection.
CHI'17.
Proceedingsofthe2017CHIConferenceonHumanFactorsinComputingSystems,Denver,CO,USA,6-11May2017.
NewYork,NY,USA:ACM,pp.
2696–2707.
Stahl,Gerry(2000).
Amodelofcollaborativeknowledge-building.
InB.
J.
Fishman&S.
F.
O'Connor-Divelbiss(eds.
):ICLS2000.
ProceedingsoftheFourthInternationalConferenceoftheLearningSciences,AnnArbor,Michigan,USA,14-17June2000.
Mahwah,NJ,USA:LawrenceErlbaumAssociates,pp.
70–77.
PrillaMichaeletal.
740Suchman,Lucy(1996).
Supportingarticulationwork.
InR.
Kling(ed.
):Computerizationandcontroversy:Valueconictsandsocialchoices.
SanDiego:AcademicPress,2nded,pp.
407–423.
Suthers,DanielD.
(2000).
Initialevidenceforrepresentationalguidanceoflearningdiscourse.
ICCE'02.
ProceedingsofInternationalConferenceonComputersinEducation,Taipei,Taiwan,21-24November2000.
Taipei,Taiwan,pp.
21–24.
Tang,Charlotte;andYunanChen(2015).
TechnologyReadinessthroughCollaborativeReection:AQualitativeCaseStudyofanUnder-resourcedNurse-ManagedClinic.
InT.
X.
Bui&R.
H.
Sprague(eds.
):HICCS2015.
Proceedingsofthe48thAnnualHawaiiInternationalConferenceonSystemSciences,Kauai,HI,USA,5-8January2015.
LosAlamitos,California;Washington;Tokyo:IEEEComputerSociety,pp.
3013–3022.
Tausczik,YlaR.
;andJamesW.
Pennebaker(2010).
ThePsychologicalMeaningofWords:LIWCandComputerizedTextAnalysisMethods.
JournalofLanguageandSocialPsychology,vol.
29,no.
1,March2010,pp.
24–54.
Thillmann,Hubertina;JosefKünsting;JoachimWirth;andDetlevLeutner(2009).
IsitMerelyaQuestionof"What"toPromptorAlso"When"toPrompt:TheRoleofPointofPresentationTimeofPromptsinSelf-RegulatedLearning.
ZeitschriftFürPdagogischePsychologie,vol.
23,no.
2,January2009,pp.
105–115.
Tigelaar,DinekeE.
H.
;DianaH.
J.
M.
Dolmans;PaulienC.
Meijer;WillemS.
deGrave;andCeesP.
M.
vanderVleuten(2008).
Teachers'InteractionsandtheirCollaborativeReectionProcessesduringPeerMeetings.
AdvancesinHealthSciencesEducation,vol.
13,no.
3,August2008,pp.
289–308.
Woerkom,Mariannevan;andMarcelCroon(2008).
Operationalisingcriticallyreectiveworkbehaviour.
PersonnelReview,vol.
37,no.
3,April2008,pp.
317–331.
Weick,KarlE.
(1995).
Sensemakinginorganizations.
ThousandOaks:SagePublications.
Zaki,MohammedJ.
(2001).
SPADE:AnEfcientAlgorithmforMiningFrequentSequences.
MachineLearning,vol.
42,no.
1–2,January2001,pp.
31–60.
Zhu,Erping(1996).
MeaningNegotiation,KnowledgeConstruction,andMentoringinaDistanceLearningCourse.
ProceedingsofSelectedResearchandDevelopmentPresentationsatthe1996NationalConventionsoftheAssociationforEducationCommunicationsandTechnology,pp.
821–844.
Publisher'sNoteSpringerNatureremainsneutralwithregardtojurisdictionalclaimsinpublishedmapsandinstitutionalafliations.
HowDoesCollaborativeReectionUnfoldinOnline.
.
.
741