EagerContentHandler. #9051 (#12077)

Fix #9051 with EagerContentHandler to replace DelayedHandler

The make the EagerContentHandler.RetainedContentLoader work efficiently this PR adjusted the buffering strategy of h1, h2 and h3 to keep and reuse a retained buffer until it is mostly full.

Also fixed several bugs in XmlConfiguration:

 + A Set could not be used with a builder style method (kind of missing feature more than a bug)
 + If a property element was used in a Set it could only be a string and the type element was ignored.
 + When trying to find a native match, the vClass local variable was overwritten with the native TYPE being tried. This affected subsequent match attempts using the wrong vClass

---------

Signed-off-by: Lachlan Roberts <lachlan.p.roberts@gmail.com>
Signed-off-by: gregw <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Lachlan Roberts <lachlan.p.roberts@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>

documentation/jetty/modules/operations-guide/pages/modules/standard.adoc

@@ -129,6 +129,21 @@ The module properties are:
 include::{jetty-home}/modules/debuglog.mod[tags=documentation]
 ----
 
+[[eager-content]]
+== Module `eager-content`
+
+The `eager-content` module installs the `org.eclipse.jetty.server.handler.EagerContentHandler` at the root of the `Handler` tree.
+
+The `EagerContentHandler` can eagerly load request content, asynchronously, before calling the next `Handler`.
+For more information see xref:programming-guide:server/http.adoc#handler-use-eager[this section].
+
+`EagerContentHandler` can eagerly load content for form uploads, multipart uploads and any request content, and you can configure it with different properties for these three cases.
+
+The module properties are:
+
+----
+include::{jetty-home}/modules/eager-content.mod[tags=documentation]
+----
 
 [[eeN-deploy]]
 == Module `{ee-all}-deploy`

documentation/jetty/modules/programming-guide/pages/server/http.adoc

@@ -1198,6 +1198,44 @@ In the example above, `ContextHandlerCollection` will try to match a request to
 NOTE: `DefaultHandler` just sends a nicer HTTP `404` response in case of wrong requests from clients.
 Jetty will send an HTTP `404` response anyway if `DefaultHandler` has not been set.
 
+[[handler-use-eager]]
+==== EagerContentHandler
+
+`EagerContentHandler` reads eagerly the HTTP request content, and invokes the next `Handler` in the `Handler` tree when the request content has been read.
+
+`EagerContentHandler` should be installed when web applications use blocking I/O to read the request content, which is the typical case for Servlet or RESTful (JAX-RS) web applications.
+
+Because the request content is read eagerly and asynchronously, the web application will never (or rarely) block while reading the request content.
+In this way, the application obtains the benefits of asynchronous I/O without forcing web application developers to use more complicated asynchronous I/O APIs.
+
+The `Handler` tree structure looks like the following:
+
+[,screen]
+----
+Server
+└── (GzipHandler) // optional
+    └── EagerContentHandler
+        └── ContextHandler /app
+            └── AppHandler
+----
+
+`EagerContentHandler` should be installed in the `Handler` tree _after_ other ``Handler``s that may modify or transform the request content, like for example the `GzipHandler`.
+
+`EagerContentHandler` eagerly reads request content in the following cases:
+
+* Form request content.
+* MultiPart request content.
+* Any other type of request content.
+
+For Form request content, `EagerContentHandler` reads the whole request content, parses it into a `Fields` object, and then invokes the next `Handler`.
+This allows web applications that use blocking API calls such as `HttpServletRequest.getParameterMap()` to avoid blocking, since they can directly use the already created `Fields` object.
+
+Similarly, for MultiPart request content, `EagerContentHandler` reads the whole request content, parses it into `MultiPartFormData.Parts`, and then invokes the next `Handler`.
+This allows web applications that use blocking API calls such as `HttpServletRequest.getParts()` to avoid blocking, since the can directly use the already created `MultiPartFormData.Parts` object.
+
+For other types of request content, `EagerContentHandler` reads and retains request content bytes up to a configurable amount, and then invokes the next `Handler`, without any further processing of the request content bytes.
+This allows web applications that use blocking API calls such as `HttpServletRequest.getInputStream()` to avoid blocking in most cases (if the request is smaller than what has been configured in `EagerContentHandler`).
+
 [[handler-use-servlet]]
 === Servlet API Handlers
 

jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/FormRequestContent.java

@@ -17,6 +17,7 @@
 import java.nio.charset.Charset;
 import java.nio.charset.StandardCharsets;
 
+import org.eclipse.jetty.http.MimeTypes;
 import org.eclipse.jetty.util.Fields;
 
 /**
@@ -32,7 +33,10 @@ public FormRequestContent(Fields fields)
 
     public FormRequestContent(Fields fields, Charset charset)
     {
-        super("application/x-www-form-urlencoded", convert(fields, charset), charset);
+        super(charset == StandardCharsets.UTF_8
+            ? MimeTypes.Type.FORM_ENCODED_UTF_8.asString()
+            : MimeTypes.Type.FORM_ENCODED.asString() + ";charset=" + charset.name().toLowerCase(),
+            convert(fields, charset), charset);
     }
 
     public static String convert(Fields fields)
@@ -48,7 +52,7 @@ public static String convert(Fields fields, Charset charset)
         {
             for (String value : field.getValues())
             {
-                if (builder.length() > 0)
+                if (!builder.isEmpty())
                     builder.append("&");
                 builder.append(encode(field.getName(), charset)).append("=").append(encode(value, charset));
             }

jetty-core/jetty-client/src/test/java/org/eclipse/jetty/client/util/TypedContentProviderTest.java

@@ -35,6 +35,7 @@
 
 import static org.hamcrest.MatcherAssert.assertThat;
 import static org.hamcrest.Matchers.containsInAnyOrder;
+import static org.hamcrest.Matchers.equalToIgnoringCase;
 import static org.junit.jupiter.api.Assertions.assertEquals;
 import static org.junit.jupiter.api.Assertions.assertNotNull;
 
@@ -57,14 +58,12 @@ public void testFormContentProvider(Scenario scenario) throws Exception
             protected void service(Request request, Response response)
             {
                 assertEquals("POST", request.getMethod());
-                assertEquals(MimeTypes.Type.FORM_ENCODED.asString(), request.getHeaders().get(HttpHeader.CONTENT_TYPE));
-                FormFields.from(request).whenComplete((fields, failure) ->
-                {
-                    assertEquals(value1, fields.get(name1).getValue());
-                    List<String> values = fields.get(name2).getValues();
-                    assertEquals(2, values.size());
-                    assertThat(values, containsInAnyOrder(value2, value3));
-                });
+                assertThat(request.getHeaders().get(HttpHeader.CONTENT_TYPE), equalToIgnoringCase(MimeTypes.Type.FORM_ENCODED_UTF_8.asString()));
+                Fields fields = FormFields.getFields(request);
+                assertEquals(value1, fields.get(name1).getValue());
+                List<String> values = fields.get(name2).getValues();
+                assertEquals(2, values.size());
+                assertThat(values, containsInAnyOrder(value2, value3));
             }
         });
 
@@ -89,7 +88,7 @@ public void testFormContentProviderWithDifferentContentType(Scenario scenario) t
         fields.put(name1, value1);
         fields.add(name2, value2);
         final String content = FormRequestContent.convert(fields);
-        final String contentType = "text/plain;charset=UTF-8";
+        final String contentType = "text/plain;charset=utf-8";
 
         start(scenario, new EmptyServerHandler()
         {

jetty-core/jetty-deploy/src/main/java/org/eclipse/jetty/deploy/providers/ContextProvider.java

@@ -24,7 +24,6 @@
 import java.nio.file.Paths;
 import java.util.ArrayList;
 import java.util.Arrays;
-import java.util.Comparator;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Locale;
@@ -305,10 +304,7 @@ else if (Supplier.class.isAssignableFrom(context.getClass()))
         initializeContextPath(contextHandler, path);
 
         if (Files.isDirectory(path))
-        {
             contextHandler.setBaseResource(ResourceFactory.of(this).newResource(path));
-            System.err.println("SET BASE RESOURCE to " + path);
-        }
 
         //TODO think of better way of doing this
         //pass through properties as attributes directly

jetty-core/jetty-http/src/main/java/org/eclipse/jetty/http/MimeTypes.java

@@ -61,7 +61,11 @@ public class MimeTypes
     public enum Type
     {
         FORM_ENCODED("application/x-www-form-urlencoded"),
+        FORM_ENCODED_UTF_8("application/x-www-form-urlencoded;charset=utf-8", FORM_ENCODED),
+        FORM_ENCODED_8859_1("application/x-www-form-urlencoded;charset=iso-8859-1", FORM_ENCODED),
+
         MESSAGE_HTTP("message/http"),
+
         MULTIPART_BYTERANGES("multipart/byteranges"),
         MULTIPART_FORM_DATA("multipart/form-data"),
 
@@ -77,6 +81,10 @@ public HttpField getContentTypeField(Charset charset)
                     return super.getContentTypeField(charset);
                 }
             },
+
+        TEXT_HTML_8859_1("text/html;charset=iso-8859-1", TEXT_HTML),
+        TEXT_HTML_UTF_8("text/html;charset=utf-8", TEXT_HTML),
+
         TEXT_PLAIN("text/plain")
             {
                 @Override
@@ -89,6 +97,9 @@ public HttpField getContentTypeField(Charset charset)
                     return super.getContentTypeField(charset);
                 }
             },
+        TEXT_PLAIN_8859_1("text/plain;charset=iso-8859-1", TEXT_PLAIN),
+        TEXT_PLAIN_UTF_8("text/plain;charset=utf-8", TEXT_PLAIN),
+
         TEXT_XML("text/xml")
             {
                 @Override
@@ -101,21 +112,14 @@ public HttpField getContentTypeField(Charset charset)
                     return super.getContentTypeField(charset);
                 }
             },
-        TEXT_JSON("text/json", StandardCharsets.UTF_8),
-        APPLICATION_JSON("application/json", StandardCharsets.UTF_8),
-
-        TEXT_HTML_8859_1("text/html;charset=iso-8859-1", TEXT_HTML),
-        TEXT_HTML_UTF_8("text/html;charset=utf-8", TEXT_HTML),
-
-        TEXT_PLAIN_8859_1("text/plain;charset=iso-8859-1", TEXT_PLAIN),
-        TEXT_PLAIN_UTF_8("text/plain;charset=utf-8", TEXT_PLAIN),
-
         TEXT_XML_8859_1("text/xml;charset=iso-8859-1", TEXT_XML),
         TEXT_XML_UTF_8("text/xml;charset=utf-8", TEXT_XML),
 
+        TEXT_JSON("text/json", StandardCharsets.UTF_8),
         TEXT_JSON_8859_1("text/json;charset=iso-8859-1", TEXT_JSON),
         TEXT_JSON_UTF_8("text/json;charset=utf-8", TEXT_JSON),
 
+        APPLICATION_JSON("application/json", StandardCharsets.UTF_8),
         APPLICATION_JSON_8859_1("application/json;charset=iso-8859-1", APPLICATION_JSON),
         APPLICATION_JSON_UTF_8("application/json;charset=utf-8", APPLICATION_JSON);
 
@@ -691,7 +695,25 @@ public static MimeTypes.Type getMimeTypeFromContentType(HttpField field)
         if (field instanceof MimeTypes.ContentTypeField contentTypeField)
             return contentTypeField.getMimeType();
 
-        return MimeTypes.CACHE.get(field.getValue());
+        String contentType = field.getValue();
+        int semicolon = contentType.indexOf(';');
+        if (semicolon >= 0)
+            contentType = contentType.substring(0, semicolon).trim();
+
+        return MimeTypes.CACHE.get(contentType);
+    }
+
+    public static String getMimeTypeAsStringFromContentType(HttpField field)
+    {
+        if (field == null)
+            return null;
+
+        assert field.getHeader() == HttpHeader.CONTENT_TYPE;
+
+        if (field instanceof MimeTypes.ContentTypeField contentTypeField)
+            return contentTypeField.getMimeType().asString();
+
+        return getBase(field.getValue());
     }
 
     /**

jetty-core/jetty-http/src/main/java/org/eclipse/jetty/http/MultiPartConfig.java

@@ -18,6 +18,7 @@
 import org.eclipse.jetty.io.Content;
 import org.eclipse.jetty.util.Attributes;
 import org.eclipse.jetty.util.Promise;
+import org.eclipse.jetty.util.resource.ResourceFactory;
 
 import static org.eclipse.jetty.http.ComplianceViolation.Listener.NOOP;
 
@@ -51,6 +52,15 @@ public Builder()
         {
         }
 
+        /**
+         * @param location the directory where parts will be saved as files.
+         */
+        public Builder location(String location)
+        {
+            location(ResourceFactory.root().newResource(location).getPath());
+            return this;
+        }
+
         /**
          * @param location the directory where parts will be saved as files.
          */
@@ -70,7 +80,7 @@ public Builder maxParts(int maxParts)
         }
 
         /**
-         * @return the maximum size in bytes of the whole multipart content, or -1 for unlimited.
+         * @param maxSize the maximum size in bytes of the whole multipart content, or -1 for unlimited.
          */
         public Builder maxSize(long maxSize)
         {
@@ -79,7 +89,7 @@ public Builder maxSize(long maxSize)
         }
 
         /**
-         * @return the maximum part size in bytes, or -1 for unlimited.
+         * @param maxPartSize  the maximum part size in bytes, or -1 for unlimited.
          */
         public Builder maxPartSize(long maxPartSize)
         {

jetty-core/jetty-http2/jetty-http2-client/src/main/java/org/eclipse/jetty/http2/client/HTTP2ClientConnectionFactory.java

@@ -84,7 +84,7 @@ private static class HTTP2ClientConnection extends HTTP2Connection implements Ca
 
         private HTTP2ClientConnection(HTTP2Client client, EndPoint endpoint, HTTP2ClientSession session, Promise<Session> sessionPromise, Session.Listener listener)
         {
-            super(client.getByteBufferPool(), client.getExecutor(), endpoint, session, client.getInputBufferSize());
+            super(client.getByteBufferPool(), client.getExecutor(), endpoint, session, client.getInputBufferSize(), -1);
             this.client = client;
             this.promise = sessionPromise;
             this.listener = listener;