LOGAN: LAWS OF LEARNING (Basic Processes) January 2004 INSTRUMENTAL CONDITIONING: Excitation Current Contingency Interdependent Variables Contents Paradigmatic/Natural/Asocial/Emotionally-Positive/Interdependent-Operation Principle IECC<>1:00. Instrumental (Thorndikian) Conditioning IB****aIA IECC<>1:10. Nature of Reinforcement in Instrucond IB****aIK IECC<>1:11. Relative effectiveness of reinf IB****aiK Systematic Analyses of Instrumental Conditioning IB****aIZ IECC<>2:00. Control Procedures for Instrcond IB****aIc IECC<>3:00. Miscellaneous Instrucond Phenomena IB****aIx Miscellaneous Analyses of Instrumental Conditioning IB****aIz Paradigmatic/Natural/Asocial/Emotionally-Neutral/Interdependent-Operation Law IECC<>4:00. Instrumental Performance as-f(N/trials) IB****bIR IECC<>4:01. Perf(IR) as-f(extended trng) IB****bnn IECC<>4:10. Within-session effects IB****bnw Systematic Analyses of Acquisition Function IB****bzA Paradigmatic/Natural/Asocial/Emotionally-Negative/Interdependent-Operation Principle IECC<>5:00. Punishment (Conflict) in Instrucond IB****cIp IECC<>5:01. Pretraining Effect of Punishment IB****cat IECC<>5:02. W/in-chain Location of PunishmentIB****cas IECC<>5:03. Gradient of Avoidance IB****cag IECC<>5:10. Effect of Partial Punishment IB****caq IECC<>5:20. R-Correlated Punishment IB****car IECC<>6:00. One-trial acq w/punishment of IR IB****caA Relevant uncoded root-abstracts Relevant(?) unabstracted/uncoded literature Principle IECC<>1:00. Instrumental (Thorndikian) Conditioning IB****aIA Reinf R w/discrete trials-->acq IR Instrumental Conditioning (Instrucond) Procedures Trial-and-Error (T&E) Learning: Reinf IR in mult-R context Locomotor Learning: Reinf approach IR. Behavior Chains: Series of (Homogeneous/heterogeneous) IR Note: May observe effect of operation on other behavior DAL Title: Audition and habit formation - - - - - - - - - - - Joh,HM'12- XXX- Pair(tone,raise-paw,food) --> Incr P(raise-paw) to tone - - Mil,S.'29+ Pair(tone,raise-paw,blow) --> decr P(raise-paw) to tone Mil,S.'29+ Above=objective def of agree/disagreeable excitants Mil,S.'29+ This 2nd type of CR need not be same as UR Mil,S.'29+ 2nd type CR probably applies only w/striated muscle Mil,S.'29+ Title: Study of Thorndike's lrng theory - - - - - - - - - Tro,MH'32+ AXXX Acq connection situation --> behavior -->pos after-effect - Tho,EL'33- hTOH Acq arm-mvmt R w/pair(tone,move-arm in sling,food) - - - - Nov,AA'34- COOH Slow acq string-pull IR but not re visual string-food cue - Tru,CK'34+ RCxH Acq "face-wash"/"scratch" IR w/escape-box to food reinf - - Lor,I.'36- DAFH No cond after 350 Pair(buzz,lift-foot,food) - - - - - - - - Woo,CB'42- xUxH Difficult to estab hetero chain if unequal biol strength - Nor,LN'52- Title: Motor cond reflexes - - - - - - - - - - - - - - - - Wyr,W.'56- OAR Title: Social lrng in chick - - - - - - - - - - - - - - - Smi,W.'57- R1 Title: 1-tr cond w/lite/tone S - - - - - - - - - - - - - - Gru,RP'70- HOO Title: Instrucond trtmt of enuresis - - - - - - - - - - - Kim,HD'70+ Rfx- Chain: Obtain levers w/face-wash/scratch, cue reinf bar - - Mor,MJ'79+ R Title: Assn w/mult outcomes - - - - - - - - - - - - - - - Res,RA'91- IECC<>1:10. Nature of Reinforcement in Instrucond IB****aIK Title: Reward value of CS (Tolman) - - - - - - - - - - - - Ada,KP'28- Title: Reward value of CS (Tolman) - - - - - - - - - - - - Ada,KP'28- RORO Cross shock grid w/reinf = explore novel maze - - - - - - - Nis,HW'30- RSPO Re tend to make IR w/punish, social reinf = empty GB - - - Loc,NM'36- DTxH Reinf w/sham feeding (w/fistula discharge) < true feeding - Hul,CL'51e Subj refuses to eat after 8 d sham feeding Hul,CL'51e RSRS Speed(IR) w/sexually-naive male O to receptive female >male She,FD'51e Above O allowed to copulate but not ejaculate in GB She,FD'51e RCPH Acq IR to access diff environ, not-f(prior trng w/food-dep) Mye,AK'54+ R Title: Food for thirsty animals - - - - - - - - - - - - - Wik,EL'54+ RSRO Males acq IR w/reinf = receptive female - - - - - - - - - - Bea,FA'56+ RNRO Acq run IR and maze lrng w/forebrain ESB reinf, sim to food Old,J.'56- RARO Maint IR w/reinf = novel GB S - - - - - - - - - - - - - - - Cha,RM'57+ mFAH Acq IR w/sucking milk reinf - - - - - - - - - - - - - - - - Mas,WA'58+ CNxO Acq run IR-->hypothal ESB + shuttle ARi w/Pair(S,same ESB) Bro,GW'59+ Title: Reinforcers and reinf - - - - - - - - - - - - - - - Bev,W.'60+ RSC Title: Reward value of male sex (Beach) - - - - - - - - - Wha,RE'60- RCPH Acq bar-press IR w/reinf = leave box, sim to reinf trng - - Wik,EL'62e dARO Instrucond w/reinf = contact w/person, incr-f(person dep) - Bac,WE'63+ Title: Attractiveness as-f(dissonance) - - - - - - - - - - Rah,SM'63- RARx For most O, priming w/self-ESB stim nec for IR w/ESB reinf Wet,MC'63- RARH Speed(IR) incr-f(prior effort required to obtain unique rf) Lew,M.'64- Title: Reinf/discrim functions of S - - - - - - - - - - - Mor,JE'65e DSR Title: Cond w/passive person reinf - - - - - - - - - - - - Sta,WC'65e MVOO Infer reinf effect of mirror-image re duration open-door IR Gal,GG'66- OSO- Incr P(distress call) w/imprinted S as reinf - - - - - - - Hof,HS'66e RARO Speed(IR) w/ESB reinf incr-f(priming ESB) - - - - - - - - - Gal,CR'69- RARO ESB priming not-f(same/diff placement priming/reinf ESB) - Gal,CR'69- Choice of electrode placement not-f(priming electrode Gal,CR'69- RAR Title: Palatable nonnutritive bulk reinf - - - - - - - - - Tay,CJ'69- RART Speed(IR) w/mult reinf/trial decr-f(inter-reinf interval) - Cil,A.'70+ SARO Acq IR w/reinf = defeat submissive O, esp w/aggressive O - Leg,R.'70- RARO Maint IR w/reinf = saccharin on dry pellet - - - - - - - - Jon,EC'71+ Title: Rewards that do not reduce needs - - - - - - - - - Eis,R.'72- RMRO Speed(IR) incr-f(N/pulses of ESB) w/priming/trng varied - - Edm,DE'74+ RNRO Re Speed(IR), reinf value of ESB is trade-off re N vs inten Edm,DE'74e r Title: Lrng/altern/extn w/sucking reward - - - - - - - - - Ams,A.'76e RDRO AMPT decr reinf effect of ESB w/only some reinf brain loci Edm,DE'77+ Title: Sunflower seed/nest paper reinf - - - - - - - - - - She,SJ'78- PS Title: Reinf w/access to pigeons - - - - - - - - - - - - - Tsa,S.'79- TARO Acq run IR w/reinf = hoarding food - - - - - - - - - - - - Lau,M.'81- RS Title: Instrucond w/sex reinf - - - - - - - - - - - - - - Eve,BJ'87e R Title: Ethanol-seeking (Ettenberg) - - - - - - - - - - - - Cza,CL'98- IECC<>1:11. Relative effectiveness of reinf IB****aiK Note: Reinf effectiveness may depend on setting operations RARO Maint IR w/reinf = automatic ESB > self-produced ESB - - - Sew,JP'60e RARH Speed(IR) w/10-sec access to food > w/30; eat faster w/10 - Bla,RW'64+ Rxxx ESB = sugar-H2O as reinf re IR/consum-R - - - - - - - - - - Gib,WE'65e RNRO Speed(IR) w/ESB reinf w/5-sec > 60-sec ITI - - - - - - - - Gal,CR'66- Speed(IR) w/weak > strong ESB altho prefer strong Gal,CR'66- Interp above re rapidly decaying motiv effect of ESB Gal,CR'66- RNRO Re Speed(IR), optimal ESB reinf w/short-duration stim - - - Bar,FE'74e oUUx W/wolf pups, Speed(IR) w/social > w/food reinf - - - - - - Fra,MG'88+ Systematic Analyses of Instrumental Conditioning IB****aIZ Issue: Is reinf nec for instrucond AXXX W/chicks, reinf-->incr P(R+), punish (confine)-->no effect Tho,EL'32- RUOH Beh w/changed qual reinf implies expectation specific reinf Lor,I.'36+ RCPH Food reinf > prevent unlrng w/immed removal from sit - - - Sew,JP'42- XXX- E=f(N/trials)((f(dep)+f(Amt(reinf)))-f(Delay(reinf)) (197) Spe,KW'56- XfX- Th: R-fdbk compared w/cond image of R, w/correction - - - - Mow,OH'60- XXX- Contiguity theory of instrumental reward cond (96) - - - - Spe,KW'60- Nonreinf of previously reinf IR produces emot/R (Rf)(98) Spe,KW'60- Rf strength =f(Rg strength), becomes anticipatory (98) Spe,KW'60- Rf contributes to gen drive, elicits incompatible R (98) Spe,KW'60- Compatible IR assoc w/Rf during PRF (100) Spe,KW'60- Implies PRAE in instrumental reward cond (100) Spe,KW'60- Title: S-incentive/S-R conting in R acq - - - - - - - - - LJo,J.'78+ X Title: Contiguity/contingency in cond - - - - - - - - - - Tho,GV'83- IECC<>2:00. Control Procedures for Instrcond IB****aIc XXXX Title: Constraints on cond (Dickinson) - - - - - - - - - - She,SJ'79- RARH Concl: Distinct odors of both reinf/nonreinf as cues - - - Voo,JW'81+ RARH Beh-systems anal of structure of appetitive beh - - - - - - Tim,W.'83- Above incl unrewarded running in straight alley Tim,W.'83- Above is-f(reduction explor/caution state) re novelty Tim,W.'83- IR is-f locomotor appetitive structure re path Tim,W.'83- Few diff between rewarded/unrewarded perf in alley Tim,W.'83- IECC<>3:00. Miscellaneous Instrucond Phenomena IB****aIx RCOH Perf climb-and-jump-in-box after being put thru w/out reinf Hun,WS'12- RDLH Instrucond facil by daily glutamic acid - - - - - - - - - - Alb,KE'44+ R P Acq bar-touch IR facil w/bar avail during ITI - - - - - - - Coo,H.'47+ RTRH Tr(n+1) Speed(IR) decr-f(required force on Tr(n)) - - - - - Tho,ME'50- Above w/short ITI; no diff w/15-min ITI Tho,ME'50- MAAH Lat(IR) not-f auditory distraction re monkey/lab sounds - - But,RA'56+ RARH Direct GB feeding after nonreinf run did not incr IR speed Ste,L.'57- Direct GB feed after nonreinf run facil acq of reinf IR Ste,L.'57- RARH Speed(IR) w/out reinf incr-f(N/Tr), not-f(delay in GB) (44) Log,FA'60- RAU- 85-db noise decr R via "protective inhib" - - - - - - - - - Bor,MK'62- DARH No acq w/forced leading to sound source, decr acq w/food - Aiv,IM'63- RARH W/chain IR1-IR2, Speed(IR2) incr w/momentary block B4 IR2 - Uye,ET'65- RARH Speed(IR) incr-f(delay after click S B4 release from SB) - Kim,HD'66+ RxxH After diffcond, acq jump-ER re S paired w/smaller/no reinf Dal,HB'71- RARH After extn, start/run Speed(IR) incr re feed Os near alley Won,PT'71+ RAAH SB acty not-f trng w/w/out reinf in GB - - - - - - - - - - Kla,WF'74- RAR1 After trng w/food or H2O dep and 2-tr/day w/food-H2O or Gra,DS'80e H2O-food reinf, neg transfer w/shift to other dep Gra,DS'80e Interp above as failure to learn specific rf expectancy Gra,DS'80e Miscellaneous Analyses of Instrumental Conditioning IB****aIz RARH Incr in Speed(IR) not true lrng; inhib of R to new Ss - - - Buy,JF'31- XXX- Image of pos/neg object in environ leads to appr/avoid (4) Ber,J.'65- Image-driven beh dominate other automatic/reflex beh (7) Ber,J.'65- XXX- Describes Pavlov's notion of rapid cond of "causal" reflex Asr,EA'74- Also Pavlov's notion of bilateral cond re voluntary R Asr,EA'74- Title: Actions and habits (Spear) - - - - - - - - - - - - Ada,CD'82+ R Title: Variables that affect serial lrng - - - - - - - - - Cap,EJ'85e XXX Title: Assoc structures in instrucond - - - - - - - - - - Col,RM'86+ R Title: Assn between discrim S and reinf - - - - - - - - - Col,RM'88+ Title: Incentive: Motiv control of IR - - - - - - - - - - Dic,A.'89+ R Title: Assn of discrim S w/R-outcome - - - - - - - - - - - Res,RA'90- R Title: Cog proc in serial lrng (Fountain) - - - - - - - - Ben,DM'99- Law IECC<>4:00. Instrumental Performance as-f(N/trials) IB****bIR Speed(IR) is incr-nec-accel-f(N/trials) RARH Log-Lat(IR) neg-accel decr-f(N/acq tr) - - - - - - - - - - Gra,CH'40+ RCPH Lat(IR) neg-accel-decr-f(N/trials) - - - - - - - - - - - - Fel,JM'47e RARH Approach IR perf incr after first reinf tr - - - - - - - - Den,MR'56+ RARH Speed(IR) incr w/trng, decr w/extn w/tr w/competing R elim Mar,MH'63+ RARH Speed(IR) incr then decr over 120 tr w/w/out delayed reinf Lud,HW'65+ RAR Title: Speed as-f(N/reinf) - - - - - - - - - - - - - - - - Sch,SE'67+ IECC<>4:01. Effect of extended trng on Speed(IR) IB****bnn RARH W/CRF for 30-tr/day, O stop running as-f(dep/ITI) - - - - - Cal,AD'56e RARH No decr in Speed(IR) w/15/30 massed tr/day for 65/42 days - Lan,DE'56+ R IR acq decr-f(effort), w/complex relation to extn/spon-rec Aik,EG'57- RART Over days, decr IR if daily massed tr w/large reinf - - - - Ken,DC'58- RART Decr in IR w/extended trng not due to detention in GB - - - Ken,DC'59- RARH Maint hi Speed(IR) w/30 tr/day for 41 days - - - - - - - - Fuc,SS'60- RART Maint IR perf w/prolonged (66-d) trng - - - - - - - - - - - Jac,DE'69- IECC<>4:10. Within-session effects IB****bnw RART N/days B4 stop hi-effort IR w/massed tr w/60% reinf > 100% Ken,DC'59- RARH Speed(IR) decr w/in-day, incr on 1st tr of next day - - - - Hil,WF'65e Above reminiscence only w/massed CRF trng Hil,WF'65e RARH Speed(IR) incr-f(tr w/in day) w/2-hr ITI, incr-decr w/<1 hr Foz,JL'66- RARH Speed(IR) decr over tr w/in session, not-f(Amt(Reinf)) - - Cra,RJ'67+ Systematic Analyses of Acquisition Function IB****azA XXX- Derives Speed/Lat(IR) from Hullian oscillation/threshold - Spe,KW'54- Assumes excit-pot sampled momentarily until R occurs Spe,KW'54- Implies Speed/Lat(IR) as-f(trials) given E=f(trials) Spe,KW'54- Above derivation assumes no competing R in the sit Spe,KW'54- XXX- Heterogeneous chain: Successive S + especially R diff (45) Spe,KW'56- Homogeneous: Both S complex + R more or less sim (45) Spe,KW'56- Speed/Amp/P(R) =f(E-threshold) w/oscillation (93) Spe,KW'56- Principle IECC<>5:00. Punishment (Conflict) in Instrucond IB****cIp . Perf(IR) decr-f(Punishment Intensity) RARH Incr Speed(IR) w/add noise punish to food reward in GB - - Dre,GC'35- CARH Sodium amytal elim avoid R in conflict sit - - - - - - - - Bai,CJ'52+ Title: Conflict beh as-f(rf S) (RBerry) - - - - - - - - - Gar,WK'53- RARH Re-acq approach in W alley after pun facil w/pretrng in G - Tay,JA'59+ RARH Decr in IR speed w/incr Inten(Pun) greater if small reinf Bow,GH'60+ Resist shock pun re P(IR) decr w/extended reinf pretrng Hol,PA'60+ RARH W/food-reinf IR, Speed decr-f(constant inten punish) (209) Log,FA'60- RARH Decr in IR w/hi shock less if gradually incr inten over tr Mil,NE'60- Decr in IR w/sudden hi shock greater w/overtrng Mil,NE'60- RARH After pun IR, no facil effect of tr w/out pun in diff alley Tay,JA'61+ RDRH After trrg w/w/out drug, amobarb incr approach to pun GB - Bar,H.'62e RARH After pun, approach toward GB incr-f(trials) - - - - - - - Eld,T.'62- RARH After reinf trng, speed(IR) decr-f(pun inten), esp near GB Kar,EB'62- RARH 9-d delayed > immed reacq of approach after conflict trng - Nob,CD'62+ Reacq approach in conflict sit same as begin in diff sit Nob,CD'62+ RARH After initial trng w/food/shock, Speed(IR) not-f(sh inten) Kar,EB'63- RARH Shock in diff GB-->decr Speed(IR) after 22-h > 1-h - - - - Sal,E.'63+ XXXX Interp pun-IR re inhib attn vs active avoid - - - - - - - - Ger,L.'64- RCPH W/discrete-tr FR, Rate(IR) decr-f(pun inten), spaced>massed Kar,EG'64- H Decr effect of punish on IR w/immed reinf decr-f(Delay(Pun) Ban,RK'65+ RART Decr effect of shock punish decr-f(Delay(Pun)) - - - - - - Bar,A.'65- RAR- Pun decr IR more w/massed than w/spaced trng - - - - - - - Fay,GC'65- RNPS Avoid place w/shock+ECS > shock or ECS alone - - - - - - - Cas,JH'66- RUPH Freq/Speed(IR) incr-f(punish inten), decr-f(punish delay) - Coh.PS'68- RARH Decr Speed(IR) w/cold H2O pun less w/gradual decr H2O temp Ham,RJ'68+ RARH Decr P(IR) w/gradual incr shock pun inten < hi inten - - - Hol,PA'68+ RARH Tend to approach GB after punish incr-decr-f(time B4 test) Bin,JS'70e RART Trng w/H2O or sucrose--shock in GB--approach re sucrose>H2O Waa,WL'71e RUPH After IR decr w/R-conting shock, Lat(IR) incr w/free shock Mye,JS'75- RVO Suppress touch-novel-object w/long-delayed shock - - - - - Sul,L.'79- oARH IR-decr w/pun incr-f(N/acq tr) w/1-d, not 4-d old chicks - Mat,GA'80+ RAR Title: Punish IR vs consum R - - - - - - - - - - - - - - Rod,WA'80+ IECC<>5:01. Pretraining Effect of Punishment IB****cat RARH Decr effect of shock pun decr-f(N/prior reinf trng tr) - - Kau,EL'49+ RARH Conflict/emotionality w/pun incr-f(Amt prior trng w/out pun Kem,JE'56+ RARH Pre-expose to intense shock incr effect of pun on approach Kur,KH'62+ RARH Prior tumbling trauma incr effect of IR punish w/shock/puff And,DC'66+ RxAx Prior shock/aversive sleep dep incr effect of mild IR pun - And,DC'66e R Title: Prior shock on app/avoid conflict - - - - - - - - - Ter,W.'67+ RARU IR w/cold H2O pun facil w/prior expose to cold H2O stress - Ric,DC'72+ RARx Prior AR trng w/shock US incr effect of pun on food-rf IR - Ter,W.'72- RARH Preexpose (un)signalled shock decr effect of punish on IR - Cap,ED'85e IECC<>5:02. W/in-chain Location of Punishment IB****cas RARH W/incr shock inten, persist w/shock in alley > after goal - Fei,AR'63+ W/sudden strong sh, persist w/shock in alley ECS; CER w/ECS > footshock - - - Ger,LK'65- IECC<>5:03. Gradient of Avoidance IB****cag RAOx Re strength-pull, approach decr-f distance, incr-f(dep) - - Bro,JS'48- Re strength-pull, avoid incr-f distance, steeper than app Bro,JS'48- RARH Shock while eating after trng w/food in alley-->no approach Mur,EJ'55+ After above, distance approach decr-f(sim of test alley) Mur,EJ'55+ After approach in sim alley, approach in initial alley Mur,EJ'55+ RARH After elim IR w/pun in GB, approach distance incr-f(dep) - Tra,MA'60e RAOx Trng w/food reinf, then shock near GB-->weaker pull near GB Mar,JG'63e RART Start Speed(IR) w/50% shock consum-R (lick) > no pun - - - Mar,B.'64+ IECC<>5:10. Effect of Partial Punishment IB****caq RARH W/food-reinf IR, Speed w/partial punish > w/contin (210) Log,FA'60- W/partial punish, Speed after pun trdecr Speed(IR) B4 shock/noise Scu,J.'71- RARH Acq pattern Speed(IR) w/CRF and altern, not irreg pun - - - Cam,PE'72e RARH Speed(IR) is-f(sequence of pun tr after nonreinf in PRF) - Kau,MC'74e R Title: Attenuate effect intermit punish - - - - - - - - - Ban,RK'76- IECC<>5:20. Correlated Punishment IB****car RARH Speed w/neg correl punish < matched partial punish (217) Log,FA'60- IECC<>6:00. One-trial acq w/punishment of IR IB****caA RVCH Acq 1-tr punish IR incr-f(30-145 d age), incl push shavings Hud,BB'50- Less 1-tr lrng if remove S concur w/shock-on Hud,BB'50- Retention 1-tr punish IR decr-f(13-140 d delay B4 test) Hud,BB'50- Note: Above 1-tr lrng specific re S in home cage context Hud,BB'50- S JS Gradual incr in P(1-tr punish IR) up to 21-d B4 test - - - Cla,R.'67- RARH Lat(IR) incr-f(1-60-hr post-1-tr-shock pun delay B4 test) - All,JD'70+ SCJS Lat(IR) after 1-tr pun incr-decr-f(delay B4 test) - - - - - Gel,A.'70e SCJS Retention 1-tr punish IR decr w/hippocamp, not thalamic ESB Lid,A.'71+ R ECS decr retentn of 1-tr pun IR only if prior expose to sit Sar,SJ'73+ RARS Elim decr effect of ECS on 1-tr pun IR if force-delay of IR Sar,SJ'75e RCPS Perf 1-tr pun IR oscillate w/peaks at 15-min/12-24-36-hr - Wan,RA'76+ Relevant uncoded root-abstracts Relevant(?) unabstracted/uncoded literature Title: T&E as type of CR - - - - - - - - - - - - - - - - - You,RE'37- Title: Conflict as-f(competing Rs) - - - - - - - - - - - - Bil,IM'50- R Title: PRF and non-R acq - - - - - - - - - - - - - - - - - Lew,DJ'58+ RAR Title: Beh study of stress (BACampbell) - - - - - - - - - Ric,DC'62- RAR Title: Reward/punish of same goal R - - - - - - - - - - - Mar,B.'63- Title: Reward/pun of same R - - - - - - - - - - - - - - - Bar,SS'64- Title: Aversive stim on conflict - - - - - - - - - - - - - Ter,W.'64- R C Title: 3 types of punish on perf - - - - - - - - - - - - - Kle,MH'65- Title: Drive in app-avoid beh - - - - - - - - - - - - - - Dar,JW'66- RAR Title: Lrng to resist punish - - - - - - - - - - - - - - - Ter,W.'67+ O Title: S as pun w/chickens - - - - - - - - - - - - - - - - Nas,RF'72- R Title: Preexposure and passive avoid - - - - - - - - - - - Mel,RL'75e R Title: R-outcome vs outcome-R assn - - - - - - - - - - - - Res,RA'92-