Can Gradient Descent get stuck in a
local minimum when training a Logistic Regression model?

Question

BlackTom AI · Accepted Answer

Consider the statement about Gradient Descent and local minima in training a Logistic Regression model. 
Option 1: True. If you said this, you’d be implying that gradient descent can get trapped in a local minimum when optimizing the Logistic Regression objective. In practice, the objective for logistic regression is convex (especially with a standardL2 regularization term), meaning any local minimum is also a global minimum. Therefore, the idea of getting stuck in a non-global local minimum is not characteristic of the standard logistic regression loss surface. 
Option 2: False. This aligns with the theoretical property that the logistic regression loss function, including common regularization terms, is convex with respect to the model parameters. A convex function has the single lowest point (global minimum), so gradient descent will converge to that point from any reasonable starting position, assuming appropriate learning rate and no ill-conditioning. 
Exploring potential confusion: if one encounters a non-convex variant of the problem (for example, a non-convex regularizer or a neural network with hidden layers), gradient-based methods can get trapped in local minima or saddle points. However, for plain Logistic Regression, the optimization landscape is convex, so local minima are not an issue in the usual sense. 
Another subtle point: even though gradient descent is guaranteed to reach the minimum in convex cases, in practice convergence speed and numerical issues can make it appear slow or stall near flat regions, but that does not constitute being stuck in a worse local minimum. 
In summary, the claim that gradient descent can get stuck in a local minimum when training standard Logistic Regression is not correct because the objective is convex, ensuring any local minimum is global. However, if someone introduces non-convex components or poor optimization settings, different behavior could occur, which is why the nuance matters.

CS 424.01-Spring 2025 Intro to Machine Learning Self-Assessment: Logistic Regression

Can Gradient Descent get stuck in a local minimum when training a Logistic Regression model?

View Explanation

Log in for full answers

Similar Questions

In a consumer society, many adults channel creativity into buying things

Economic stress and unpredictable times have resulted in a booming industry for self-help products

People born without creativity never can develop it

A product has a selling price of $20, a contribution margin ratio of 40% and fixed cost of $120,000. To make a profit of $30,000. The number of units that must be sold is: Type the number without $ and a comma. Eg: 20000

Which of the following statement regarding cost is correct:

Under the assumptions used in cost-volume-profit analysis, as the activity increases:

The CVP and break-even analysis are supported by a range of important assumptions. Which of the following is not an assumption of the CVP:

The break-even point is that at which:

More Practical Tools for Students Powered by AI Study Helper

Homework AI Solver

Stylized AI Paper Writer

Plagiarism Checker Assistant

Citation AI Academic Writing Tool

In-Class Translation Assistant

AI Note Generator

AI Quiz Answers

Past Exam Questions from University Test Bank

Smart Practice Assistant

Adaptive Practice

Making Your Study Simpler