A Month Of Saturdays Playlist : August 2013
A Month Of Saturdays Playlist : July 2013
A Month Of Saturdays Playlist : June 2013
A Month Of Saturdays Playlist : May 2013
"Random Access Memories" joins Mylo’s "Destroy Rock & Roll" and Chromeo’s "Business Casual" in my top 3 "albums that blow you away"
Machine Learning 2 : Reinforcement Learning vs. Supervised Learning
"Reinforcement learning (RL) and supervised learning are usually portrayed as distinct methods of learning from experience. RL methods are often applied to problems involving sequential dynamics and optimization of a scalar performance objective, with online exploration of the effects of actions. Supervised learning methods, on the other hand, are frequently used for problems involving static input-output mappings and minimization of a vector error signal, with no explicit dependence on how training examples are gathered. As discussed by Barto and Dietterich (this volume), the key feature distinguishing RL and supervised learning is whether training information from the environment serves as an evaluation signal or as an error signal…"
[Source : Supervised Actor-Critic Reinforcement Learning - paper discussing problems where both kinds of feedback ( evaluation signal/error signals ) are available to a learning system at the same time) ]
“Having discussed key differences between reinforcement learning and supervised learning, the question arises as to whether these differences are fundamental or merely superficial differences that can be eliminated with suitable problem reformulation.”
Machine Learning 1 : Overview
These notes are my own observations and questions from my study of the material on Stanford’s Engineering Everywhere Machine Learning Course
Machine Learning : a scientific discipline concerned with the design and development of algorithms that allow machines (computers) to make decisions or even evolve behaviors based on collection of empirical data.
The algorithms of machine learning fall into the following primary categories ( source : Wikipedia ) :
- Unsupervised Learning : the problem of trying to find hidden structure in unlabeled data
- Supervised Learning : inferring a function from labeled training data
- Reinforcement Learning : concerned with how an agent ought to take actions in an environment so as to maximize some notion of cumulative reward
( There are other classifications of algorithms, however these three classes are the major types discussed in the reference material I’ve encountered so far )
Supervised vs. Unsupervised Learning
We can see that Supervised Learning and Unsupervised Learning are mutually exclusive categories if the training data is either all labelled or all unlabelled.
What does it mean to say data is labelled ?
Alternative definitions of the Supervised and Unsupervised learning from a Stanford University glossary help us see what it means to say data is labelled:
- Supervised Learning: Learning techniques used to learn the relationship between independent attributes and a designated dependent attribute (the label)
- Unsupervised Learning : Learning techniques that group instances without a pre-specified dependent attribute.
We can say from these definitions that training data is said to be labelled when the data being analysed is comprised of pairs consisting of a set of independent input attributes and a dependent output attribute (the label), whereas with unlabeled data there is no pre-specified dependent attribute.
When a dataset is unlabeled, we may wish to use an Unsupervised Learning algorithm to find patterns within the structure of this data.
When a dataset contains labels, the label is often referred to as the “correct” or desired answer, and may wish to use a Supervised Learning algorithm to infer a function which predicts this “right answer” given a set of input attributes.
"To describe the Supervised Learning algorithm slightly more formally, our goal is, given a training set, to learn a function h : X → Y so that h(x) is a “good” predictor for the corresponding value of y. For historical reasons, this function h is called a hypothesis.”
( here X denotes the space of input values, and Y the spaces of output values.)
[Source: CS229 Lecture Notes - 1 ]
( The terms predict and predictor are highlighted in the above text as we will shortly attempt to define what we mean by these terms in more detail. )
Examples of Unsupervised/Supervised Learning
If we had a dataset consisting of the location coordinates of houses within a given city, we could say this is an unlabelled data set and we could use an Unsupervised Learning algorithm to find clusters of homes in the city.
If our dataset included the price of each house in addition to the location attributes, we could use a Supervised Learning algorithm to look for a function which predicted the price of the house given its location.
Reinforcement Learning : concerned with how an agent ought to take actions in an environment so as to maximize some notion of cumulative reward“Somewhere between supervised and unsupervised learning is reinforcement learning. Reinforcement learning is primarily used in control systems that might take a sequence of steps over a period of time to arrive at a desired outcome. We may know what that outcome should be, but we may not know whether any particular step is a good one (advances us toward the goal) or a bad one. Like unsupervised learning, reinforcement learning systems are trained with unlabeled data (the “right answer” is not given), but are then “rewarded” for making positive progress or achieving a goal, or “punished” for poor progress or an undesirable outcome. “Positive progress” may be counter-intuitive if it is sometimes necessary to proceed in ways that temporarily seem wrong in order to avoid an obstacle, and reinforcement learning has a general “credit assignment problem” across all the steps in a sequence when evaluating how to apportion blame for an undesirable outcome to the intermediate steps that achieved it. It would be incorrect to assume, for example, that applying the brakes caused crashes in autonomous cars simply because every failed self- driving attempt terminated with the brakes applied as the control system tried to avoid the crash. Instead, blame (negative credit) should be levied to the actions that led, perhaps much earlier, to the crash”
[ Source : “Machine Learning, Cognition, and Big Data
by Steve Oberlin, Senior Vice President and Distinguished Engineer, CA Technologies” ]
A Month Of Saturdays Playlist : April 2013
exfm-oauth-api-proxy : A proxy to Ex.Fm’s API supporting OAuth2 Authentication - currently deployed to exfmproxy.socialsignin.org
(source code at https://github.com/socialsignin/exfm-oauth-api-proxy )
Some time ago I wanted to provide “login with Ex.Fm” functionality on a number of my websites and to access authenticated methods of Ex.Fm’s API on behalf of my logged in users.
I started work on a spring-social-exfm module that acts as a Java client, but as Ex.Fm’s API does not currently support OAuth this module was limited in terms of it’s compatibility with the Spring Social framework, requiring a user to explicitly provide a username and password to access authenticated API methods.
I started experimenting with creating a proxy to Ex.Fm’s API that supports OAuth2 authentication, which would then mean I could develop a Spring Social module for Ex.Fm which could be used as an OAuth client should Ex.Fm implement OAuth in their API in future.
The result is exfm-oauth-api-proxy - currently deployed to exfmproxy.socialsignin.org - a simple webapp which delegates requests for a select subset of Ex.Fm’s API methods to Ex.Fm itself, but which also translates OAuth2 authenticated requests into username/password encoded requests before proxying the requests onwards.
This proxy is currently being used to support the “login/connect with exfm” use-cases at the following sites:
The exfm-oauth-api-proxy webapp was created from the sparklr2 sample in the spring-security-oauth project and provides the following OAuth2 endpoints that can be used to configure spring-social-exfm so it can be used within Spring Social enabled applications:
Client Ids, Secrets and Redirect Uris are configured within the
element in spring-config.xml
A Month Of Saturdays Playlist : March 2013
This webapp demonstrates using spring-social-security to delegate authentication to third party providers.
Other modules showcased are:
SocialSignin provider wrappers are used around the corresponding Spring Social modules to provide auto-registration of Spring Social components through component scanning and to provide an abstraction layer allowing the application to deal with API clients for a number of common use-cases instead of interacting with lower-level connection API
Users can create accounts with http://socialsignin.org by logging in with one of the available providers and once logged in can additionally connect their accounts with the remaining providers.