Skip to content

Commit 2bd8b32

Browse files
NiharikaduttaNiharikadutta
authored
Prep 0.11.0 release (#507)
1 parent c340f4a commit 2bd8b32

File tree

5 files changed

+105
-4
lines changed

5 files changed

+105
-4
lines changed

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939
<tbody align="center">
4040
<tr>
4141
<td >2.3.*</td>
42-
<td rowspan=6><a href="https://github.com/dotnet/spark/releases/tag/v0.10.0">v0.10.0</a></td>
42+
<td rowspan=6><a href="https://github.com/dotnet/spark/releases/tag/v0.11.0">v0.11.0</a></td>
4343
</tr>
4444
<tr>
4545
<td>2.4.0</td>

‎benchmark/scala/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
<modelVersion>4.0.0</modelVersion>
44
<groupId>com.microsoft.spark</groupId>
55
<artifactId>microsoft-spark-benchmark</artifactId>
6-
<version>0.10.0</version>
6+
<version>0.11.0</version>
77
<inceptionYear>2019</inceptionYear>
88
<properties>
99
<encoding>UTF-8</encoding>

‎docs/release-notes/0.11/release-0.11.md

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# .NET for Apache Spark 0.11 Release Notes
2+
3+
### New Features and Improvements
4+
5+
* Ability to pass and return [corefxlab](https://github.com/dotnet/corefxlab) DataFrames to UDF APIs ([#277](https://github.com/dotnet/spark/pull/277))
6+
* Support for ML TF-IDF (Term frequency-inverse document frequency) feature vectorization method ([#394](https://github.com/dotnet/spark/pull/394))
7+
* Support for TimestampType in `DataFrame.Collect()`, `CreateDataFrame` and UDFs ([#428](https://github.com/dotnet/spark/pull/428))
8+
* Support for Broadcast Variables ([#414](https://github.com/dotnet/spark/pull/414))
9+
* Support for ML feature Word2Vec ([#491](https://github.com/dotnet/spark/pull/491))
10+
* Streamline logging when there is a failure ([#439](https://github.com/dotnet/spark/pull/439))
11+
12+
13+
### Breaking Changes
14+
15+
* `SparkSession.Catalog` is changed from a method to a property ([#508](https://github.com/dotnet/spark/pull/508))
16+
17+
### Compatibility
18+
19+
#### Backward compatibility
20+
21+
The following table describes the oldest version of the worker that the current version is compatible with, along with new features that are incompatible with the worker.
22+
23+
<table>
24+
<thead>
25+
<tr>
26+
<th>Oldest compatible Microsoft.Spark.Worker version</th>
27+
<th>Incompatible features</th>
28+
</tr>
29+
</thead>
30+
<tbody align="center">
31+
<tr>
32+
<td rowspan=4>v0.9.0</td>
33+
<td>DataFrame with Grouped Map UDF <a href="https://github.com/dotnet/spark/pull/277">(#277)</a></td>
34+
</tr>
35+
<tr>
36+
<td>DataFrame with Vector UDF <a href="https://github.com/dotnet/spark/pull/277">(#277)</a></td>
37+
</tr>
38+
<tr>
39+
<td>Support for Broadcast Variables <a href="https://github.com/dotnet/spark/pull/414">(#414)</a></td>
40+
</tr>
41+
<tr>
42+
<td>Support for TimestampType <a href="https://github.com/dotnet/spark/pull/428">(#428)</a></td>
43+
</tr>
44+
</tbody>
45+
</table>
46+
47+
#### Forward compatibility
48+
49+
The following table describes the oldest version of .NET for Apache Spark release that the current worker is compatible with.
50+
51+
<table>
52+
<thead>
53+
<tr>
54+
<th>Oldest compatible .NET for Apache Spark release version</th>
55+
</tr>
56+
</thead>
57+
<tbody align="center">
58+
<tr>
59+
<td>v0.9.0</td>
60+
</tr>
61+
</tbody>
62+
</table>
63+
64+
### Supported Spark Versions
65+
66+
The following table outlines the supported Spark versions along with the microsoft-spark JAR to use with:
67+
68+
<table>
69+
<thead>
70+
<tr>
71+
<th>Spark Version</th>
72+
<th>microsoft-spark JAR</th>
73+
</tr>
74+
</thead>
75+
<tbody align="center">
76+
<tr>
77+
<td>2.3.*</td>
78+
<td>microsoft-spark-2.3.x-0.11.0.jar</td>
79+
</tr>
80+
<tr>
81+
<td>2.4.0</td>
82+
<td rowspan=5>microsoft-spark-2.4.x-0.11.0.jar</td>
83+
</tr>
84+
<tr>
85+
<td>2.4.1</td>
86+
</tr>
87+
<tr>
88+
<td>2.4.3</td>
89+
</tr>
90+
<tr>
91+
<td>2.4.4</td>
92+
</tr>
93+
<tr>
94+
<td>2.4.5</td>
95+
</tr>
96+
<tr>
97+
<td>2.4.2</td>
98+
<td><a href="https://github.com/dotnet/spark/issues/60">Not supported</a></td>
99+
</tr>
100+
</tbody>
101+
</table>

‎eng/Versions.props

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
<?xml version="1.0" encoding="utf-8"?>
22
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
33
<PropertyGroup>
4-
<VersionPrefix>0.10.0</VersionPrefix>
4+
<VersionPrefix>0.11.0</VersionPrefix>
55
<PreReleaseVersionLabel>prerelease</PreReleaseVersionLabel>
66
<RestoreSources>
77
$(RestoreSources);

‎src/scala/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
<version>${microsoft-spark.version}</version>
88
<properties>
99
<encoding>UTF-8</encoding>
10-
<microsoft-spark.version>0.10.0</microsoft-spark.version>
10+
<microsoft-spark.version>0.11.0</microsoft-spark.version>
1111
</properties>
1212

1313
<modules>

0 commit comments

Comments
 (0)