It is well established that learning can occur without external feedback, yet normative reinforcement learning theories have difficulties explaining such instances of learning. Here, we propose that human observers are capable of generating their own feedback signals by monitoring internal decision variables. We investigated this hypothesis in a visual perceptual learning task using fMRI and confidence reports as a measure for this monitoring process. Employing a novel computational model in which learning is guided by confidence-based reinforcement signals, we found that mesolimbic brain areas encoded both anticipation and prediction error of confidence—in remarkable similarity to previous findings for external reward-based feedback. We demonstrate that the model accounts for choice and confidence reports and show that the mesolimbic confidence prediction error modulation derived through the model predicts individual learning success. These results provide a mechanistic neurobiological explanation for learning without external feedback by augmenting reinforcement models with confidence-based feedback.
Each year, up to 15% of the world's population experience symptoms of an influenza infection, also commonly known as flu. The most common culprit is a strain of the virus called influenza type A subtype H3N2. One reason that so many people become infected each year is that this virus evolves rapidly. Within a few years, proteins on the surface of the virus known as antigens become less recognizable to the immune system of a person who has been previously infected. This means that the person can become ill with the virus again because their immune system cannot mount an effective response to the evolved virus strain.
Influenza virus strains evolve rapidly because their genetic material accumulates mutations quickly. Although some of these mutations are beneficial to the virus, other mutations are harmful and reduce the ability of the virus to spread. Sometimes beneficial mutations may occur alongside harmful ones, but it is not known how the harmful mutations affect the evolution of the virus.
Here, Koelle and Rasmussen used computer models of H3N2 influenza to examine the effect of harmful mutations on the evolution of this virus population. The models show that harmful mutations limit how quickly the antigens can evolve. Also, the presence of these harmful mutations effectively acts as a sieve: they allow only large changes in the antigens to establish in the virus population.
The models suggest that there are three routes by which large changes in the antigens on H3N2 viruses may occur. The first is by a single mutation that has a big effect on the antigens in viruses that only carry a few harmful mutations, but these large mutations would not happen very often. Another route may be through more common mutations that have only a small or moderate benefit, which would allow the virus to become more common in the population before it acquires a beneficial mutation with a much greater effect. The third possibility is that a large beneficial mutation may arise in viruses that have many harmful mutations. These harmful mutations may initially limit the ability of the virus to spread, but over time, some of these harmful mutations may then be lost.
Koelle and Rasmussen found that the computer models could recreate the patterns of virus evolution that have been observed in real strains of H3N2. Researchers use predictions of influenza evolution to help them decide which virus strains should be included in flu vaccines each year. Koelle and Rasmussen findings indicate that harmful mutations should be considered when making these predictions.
TNF Receptor Associated Factor 2 (TRAF2) is an adaptor protein that transduces signals following ligation of certain cytokine receptors including those binding TNF. It was first identified together with TRAF1 as a component of TNF receptor-2 and then TNF receptor-1 (TNFR1) signalling complexes (Rothe et al., 1994; Shu et al., 1996). TRAF2, like most other TRAFs, contains a RING domain, several zinc fingers, a TRAF-N, and a conserved TRAF-C domain which is responsible for oligomerisation and receptor binding through its MATH region (Takeuchi et al., 1996; Uren and Vaux, 1996).
RING domains are nearly always associated with ubiquitin E3 ligase activity (Shi and Kehrl, 2003) and TRAF2 can promote ubiquitylation of RIPK1 in TNFR1 signalling complexes (TNFR1-SC) (Wertz et al., 2004). However TRAF2 recruits E3 ligases such as cIAPs to TNFR1-SC and these have also been shown to be able to ubiquitylate RIPK1 and regulate TNF signalling (Dynek et al., 2010; Mahoney et al., 2008; Varfolomeev et al., 2008; Vince et al., 2009). This makes it difficult to unambiguously determine the role of the E3 ligase activity of TRAF2.
Activation of JNK and NF-κB by TNF is reduced in cells from Traf2-/- mice while only JNK signalling was affected in lymphocytes from transgenic mice that express a dominant negative (DN) form of TRAF2 that lacks the RING domain (Lee et al., 1997; Yeh et al., 1997). Traf2-/-Traf5-/- mouse embryonic fibroblasts (MEFs) have a pronounced defect in activation of NF-κB by TNF, suggesting that absence of TRAF2 can be compensated by TRAF5 (Tada et al., 2001). Although activation of NF-κB was restored in Traf2-/-Traf5-/- cells by re-expression of wild type TRAF2, it was not restored when the cells were reconstituted with TRAF2 point mutants that could not bind cIAPs (Vince et al., 2009; Zhang et al., 2010). These data, together with a wealth of different lines of evidence showing that cIAPs are critical E3 ligases required for TNF-induced canonical NF-κB (Blackwell et al., 2013; Haas et al., 2009; Silke, 2011), support the idea that the main function of TRAF2 in TNF-induced NF-κB is to recruit cIAPs to the TNFR1-SC. However, it remains possible that the RING of TRAF2 plays another function, such as in activating JNK and protecting cells from TNF-induced cell death (Vince et al., 2009; Zhang et al., 2010). Furthermore it has been shown that TRAF2 can K48-ubiquitylate caspase-8 to set the threshold for TRAIL or Fas induced cell death (Gonzalvez et al., 2012). Moreover, TRAF2 inhibits non-canonical NF-κB signalling (Grech et al., 2004; Zarnegar et al., 2008) and this function requires the RING domain of TRAF2 to induce proteosomal degradation of NIK (Vince et al., 2009). However, structural and in vitro analyses indicate that, unlike TRAF6, the RING domain of TRAF2 is unable to bind E2 conjugating enzymes (Yin et al., 2009), and is therefore unlikely to have intrinsic E3 ligase activity.
Sphingosine-1-phosphate (S1P) is a pleiotropic sphingolipid mediator that regulates proliferation, differentiation, cell trafficking and vascular development (Pitson, 2011). S1P is generated by sphingosine kinase 1 and 2 (SPHK1 and SPHK2) (Kohama et al., 1998; Liu et al., 2000). Extracellular S1P mainly acts by binding to its five G protein-coupled receptors S1P1-5 (Hla and Dannenberg, 2012). However, some intracellular roles have been suggested for S1P, including the blocking of the histone deacetylases, HDAC1/2 (Hait et al., 2009) and the induction of apoptosis through interaction with BAK and BAX (Chipuk et al., 2012).
Recently, it was suggested that the RING domain of TRAF2 requires S1P as a co-factor for its E3 ligase activity (Alvarez et al., 2010). Alvarez and colleagues proposed that SPHK1 but not SPHK2 is activated by TNF and phosphorylates sphingosine to S1P which in turn binds to the RING domain of TRAF2 and serves as an essential co-factor that was missing in the experiments of Yin et al. Alvarez and colleagues, observed that in the absence of SPHK1, TNF-induced NF-κB activation was completely abolished.
Although we know a lot about TRAF2, there are still important gaps particularly with regard to cell type specificity and in vivo function of TRAF2. Moreover, despite the claims that SPHK1 and its product, S1P, are required for TRAF2 to function as a ubiquitin ligase, the responses of Traf2-/- and Sphk1-/- cells to TNF were not compared. Therefore, we undertook an analysis of TRAF2 and SPHK1 function in TNF signalling in a number of different tissues.
Surprisingly, we found that neither TRAF2 nor SPHK1 are required for TNF mediated canonical NF-κB and MAPK signalling in macrophages. However, MEFs, murine dermal fibroblasts (MDFs) and keratinocytes required TRAF2 but not SPHK1 for full strength TNF signalling. In these cell types, absence of TRAF2 caused a delay in TNF-induced activation of NF-κB and MAPK, and sensitivity to killing by TNF was increased. Absence of TRAF2 in keratinocytes in vivo resulted in psoriasis-like epidermal hyperplasia and skin inflammation. Unlike TNF-dependent genetic inflammatory skin conditions, such as IKK2 epidermal knock-out (Pasparakis et al., 2002) and the cpdm mutant (Gerlach et al., 2011), the onset of inflammation was only delayed, and not prevented by deletion of TNF. This early TNF-dependent inflammation is caused by excessive apoptotic but not necroptotic cell death and could be prevented by deletion of Casp8. We observed constitutive activation of NIK and non-canonical NF-κB in Traf2-/- keratinocytes which caused production of inflammatory cytokines and chemokines. We were able to reverse this inflammatory phenotype by simultaneously deleting both Tnf and Nfkb2 genes. Our results highlight the important role TRAF2 plays to protect keratinocytes from cell death and to down-regulate inflammatory responses and support the idea that intrinsic defects in keratinocytes can initiate psoriasis-like skin inflammation.
- BRK, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article
- HK, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article
- KS, Conception and design, Drafting or revising the article
For email@example.com (KS)
The authors declare that no competing interests exist.
- National Institute on Deafness and Other Communication Disorders
- R01 DC013280
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
You are viewing the most recent version of this article.
- Previous version (November 17, 2015)
- Previous version (November 14, 2015)
- Received August 27, 2015.
- Accepted November 12, 2015.
- Published November 14, 2015.
Mani Ramaswami, Reviewing editor, Trinity College Dublin, Ireland