pyspark.sql.Column#

class pyspark.sql.Column(jc)[source]#

A column in a DataFrame.

New in version 1.3.0.

Changed in version 3.4.0: Supports Spark Connect.

Examples

Column instances can be created by

>>> df = spark.createDataFrame(
...      [(2, "Alice"), (5, "Bob")], ["age", "name"])

Select a column out of a DataFrame >>> df.name Column<’name’> >>> df[“name”] Column<’name’>

Create from an expression

>>> df.age + 1
Column<...>
>>> 1 / df.age
Column<...>

Methods

`alias`(alias, *kwargs)	Returns this column aliased with a new name or names (in the case of expressions that return more than one column, such as explode).
`asc`()	Returns a sort expression based on the ascending order of the column.
`asc_nulls_first`()	Returns a sort expression based on ascending order of the column, and null values return before non-null values.
`asc_nulls_last`()	Returns a sort expression based on ascending order of the column, and null values appear after non-null values.
`astype`(dataType)	`astype()` is an alias for `cast()`.
`between`(lowerBound, upperBound)	Check if the current column's values are between the specified lower and upper bounds, inclusive.
`bitwiseAND`(other)	Compute bitwise AND of this expression with another expression.
`bitwiseOR`(other)	"
`bitwiseXOR`(other)	Compute bitwise XOR of this expression with another expression.
`cast`(dataType)	Casts the column into type `dataType`.
`contains`(other)	Contains the other element.
`desc`()	Returns a sort expression based on the descending order of the column.
`desc_nulls_first`()	Returns a sort expression based on the descending order of the column, and null values appear before non-null values.
`desc_nulls_last`()	Returns a sort expression based on the descending order of the column, and null values appear after non-null values.
`dropFields`(*fieldNames)	An expression that drops fields in `StructType` by name.
`endswith`(other)	String ends with.
`eqNullSafe`(other)	Equality test that is safe for null values.
`getField`(name)	An expression that gets a field by name in a `StructType`.
`getItem`(key)	An expression that gets an item at position `ordinal` out of a list, or gets an item by key out of a dict.
`ilike`(other)	SQL ILIKE expression (case insensitive LIKE).
`isNaN`()	True if the current expression is NaN.
`isNotNull`()	True if the current expression is NOT null.
`isNull`()	True if the current expression is null.
`isin`(*cols)	A boolean expression that is evaluated to true if the value of this expression is contained by the evaluated values of the arguments.
`like`(other)	SQL like expression.
`name`(alias, *kwargs)	`name()` is an alias for `alias()`.
`otherwise`(value)	Evaluates a list of conditions and returns one of multiple possible result expressions.
`over`(window)	Define a windowing column.
`rlike`(other)	SQL RLIKE expression (LIKE with Regex).
`startswith`(other)	String starts with.
`substr`(startPos, length)	Return a `Column` which is a substring of the column.
`try_cast`(dataType)	This is a special version of cast that performs the same operation, but returns a NULL value instead of raising an error if the invoke method throws exception.
`when`(condition, value)	Evaluates a list of conditions and returns one of multiple possible result expressions.
`withField`(fieldName, col)	An expression that adds/replaces a field in `StructType` by name.