python - Tensorflow - adding L2 regularization loss simple example -


i familiar machine learning, learning tensorflow on own reading slides universities. below i'm setting loss function linear regression 1 feature. i'm adding l2 loss total loss, not sure if i'm doing correctly:

# regularization reg_strength = 0.01  # create loss function. tf.variable_scope("linear-regression"):     w    = tf.get_variable("w", shape=(1, 1), initializer=tf.contrib.layers.xavier_initializer())     b    = tf.get_variable("b", shape=(1,), initializer=tf.constant_initializer(0.0))     yhat = tf.matmul(x, w) + b      error_loss = tf.reduce_sum(((y - yhat)**2)/number_of_examples)     #reg_loss   = reg_strength * tf.nn.l2_loss(w)   # reg 1     reg_loss   = reg_strength * tf.reduce_sum(w**2) # reg 2     loss       = error_loss + reg_loss  # set optimizer. opt_operation = tf.train.gradientdescentoptimizer(0.001).minimize(loss) 

my specific questions are:

  1. i have 2 lines (commented reg 1 , reg 2) compute l2 loss of weight w. line marked reg 1 uses tensorflow built-in function. these 2 l2 implementations equivalent?

  2. am adding regularization loss reg_loss correctly final loss function?

almost

according l2loss operation code

output.device(d) = (input.square() * static_cast<t>(0.5)).sum(); 

it multiplies 0.5 (or in other words divides 2)


Comments

Popular posts from this blog

javascript - Thinglink image not visible until browser resize -

firebird - Error "invalid transaction handle (expecting explicit transaction start)" executing script from Delphi -

mongodb - How to keep track of users making Stripe Payments -