Package org.apache.spark.ml
Class PipelineStage
Object
org.apache.spark.ml.PipelineStage
- All Implemented Interfaces:
Serializable
,org.apache.spark.internal.Logging
,Params
,Identifiable
- Direct Known Subclasses:
Estimator
,Transformer
public abstract class PipelineStage
extends Object
implements Params, org.apache.spark.internal.Logging
A stage in a pipeline, either an
Estimator
or a Transformer
.- See Also:
-
Nested Class Summary
Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging
org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionabstract PipelineStage
Creates a copy of this instance with the same UID and some extra params.Param<?>[]
params()
Returns all params sorted by their names.abstract StructType
transformSchema
(StructType schema) Check transform validity and derive the output schema from the input schema.Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.spark.ml.util.Identifiable
toString, uid
Methods inherited from interface org.apache.spark.internal.Logging
initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext
Methods inherited from interface org.apache.spark.ml.param.Params
clear, copyValues, defaultCopy, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, onParamChange, set, set, set, setDefault, setDefault, shouldOwn
-
Constructor Details
-
PipelineStage
public PipelineStage()
-
-
Method Details
-
copy
Description copied from interface:Params
Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. SeedefaultCopy()
. -
params
Description copied from interface:Params
Returns all params sorted by their names. The default implementation uses Java reflection to list all public methods that have no arguments and returnParam
. -
transformSchema
Check transform validity and derive the output schema from the input schema.We check validity for interactions between parameters during
transformSchema
and raise an exception if any parameter value is invalid. Parameter value checks which do not depend on other parameters are handled byParam.validate()
.Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.
- Parameters:
schema
- (undocumented)- Returns:
- (undocumented)
-