github-repos/config
1
0
Fork 0
mirror of https://github.com/lightbend/config.git synced 2025-03-23 07:40:25 +08:00
Code Issues Packages Projects Releases Wiki Activity

Implement Config.checkValid() to check against a reference config

Browse Source 
This allows verifying that a config has all the keys it's supposed
to have, and also that they have plausible value types. It uses
a reference config as a kind of very loose schema.

Another benefit is that it can give users a big batch of error
messages at once, rather than one at a time via exceptions
when config settings are used.

Because the reference config is not really a schema, users of
checkValid() may have to tune its behavior by removing some
things from the reference config used for validation, or
ignoring certain errors, or doing additional validation
on their own. But checkValid() is a good basic validator
for all your simple settings and if you have complex
settings you can write additional code to support them
while still using checkValid() for the simple ones.
This commit is contained in:
Havoc Pennington 2011-11-28 01:14:31 -05:00
parent 6692dba893
commit 789930cd8e
8 changed files with 513 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -218,12 +218,6 @@ value just disappear if the substitution is not found.
Here are some features that might be nice to add.
- "Type consistency": if a later config file changes the type of a
value from its type in `myapp-reference.conf` then complain
at parse time.
Right now if you set the wrong type, it will only complain
when the app tries to use the setting, not when the config
file is loaded.
- "myapp.d directory": allow parsing a directory. All `.json`,
`.properties` and `.conf` files should be loaded in a
deterministic order based on their filename.

94
config/src/main/java/com/typesafe/config/Config.java
View File

@ -98,19 +98,19 @@ public interface Config extends ConfigMergeable {
* <code>Config</code> as the root object, that is, a substitution
* <code>${foo.bar}</code> will be replaced with the result of
* <code>getValue("foo.bar")</code>.
*
*
* <p>
* This method uses {@link ConfigResolveOptions#defaults()}, there is
* another variant {@link Config#resolve(ConfigResolveOptions)} which lets
* you specify non-default options.
*
*
* <p>
* A given {@link Config} must be resolved before using it to retrieve
* config values, but ideally should be resolved one time for your entire
* stack of fallbacks (see {@link Config#withFallback}). Otherwise, some
* substitutions that could have resolved with all fallbacks available may
* not resolve, which will be a user-visible oddity.
*
*
* <p>
* <code>resolve()</code> should be invoked on root config objects, rather
* than on a subtree (a subtree is the result of something like
@ -120,14 +120,14 @@ public interface Config extends ConfigMergeable {
* from the root. For example, if you did
* <code>config.getConfig("foo").resolve()</code> on the below config file,
* it would not work:
*
*
* <pre>
* common-value = 10
* foo {
* whatever = ${common-value}
* }
* </pre>
*
*
* @return an immutable object with substitutions resolved
* @throws ConfigException.UnresolvedSubstitution
* if any substitutions refer to nonexistent paths
@ -146,6 +146,90 @@ public interface Config extends ConfigMergeable {
*/
Config resolve(ConfigResolveOptions options);
/**
* Validates this config against a reference config, throwing an exception
* if it is invalid. The purpose of this method is to "fail early" with a
* comprehensive list of problems; in general, anything this method can find
* would be detected later when trying to use the config, but it's often
* more user-friendly to fail right away when loading the config.
*
* <p>
* Using this method is always optional, since you can "fail late" instead.
*
* <p>
* You must restrict validation to paths you "own" (those whose meaning are
* defined by your code module). If you validate globally, you may trigger
* errors about paths that happen to be in the config but have nothing to do
* with your module. It's best to allow the modules owning those paths to
* validate them. Also, if every module validates only its own stuff, there
* isn't as much redundant work being done.
*
* <p>
* If no paths are specified in <code>checkValid()</code>'s parameter list,
* validation is for the entire config.
*
* <p>
* If you specify paths that are not in the reference config, those paths
* are ignored. (There's nothing to validate.)
*
* <p>
* Here's what validation involves:
*
* <ul>
* <li>All paths found in the reference config must be present in this
* config or an exception will be thrown.
* <li>
* Some changes in type from the reference config to this config will cause
* an exception to be thrown. Not all potential type problems are detected,
* in particular it's assumed that strings are compatible with everything
* except objects and lists. This is because string types are often "really"
* some other type (system properties always start out as strings, or a
* string like "5ms" could be used with {@link #getMilliseconds}). Also,
* it's allowed to set any type to null or override null with any type.
* <li>
* Any unresolved substitutions in this config will cause a validation
* failure; both the reference config and this config should be resolved
* before validation. If the reference config is unresolved, it's a bug in
* the caller of this method.
* </ul>
*
* <p>
* If you want to allow a certain setting to have a flexible type (or
* otherwise want validation to be looser for some settings), you could
* either remove the problematic setting from the reference config provided
* to this method, or you could intercept the validation exception and
* screen out certain problems. Of course, this will only work if all other
* callers of this method are careful to restrict validation to their own
* paths, as they should be.
*
* <p>
* If validation fails, the thrown exception contains a list of all problems
* found. See {@link ConfigException.ValidationFailed#problems}. The
* exception's <code>getMessage()</code> will have all the problems
* concatenated into one huge string, as well.
*
* <p>
* Again, <code>checkValid()</code> can't guess every domain-specific way a
* setting can be invalid, so some problems may arise later when attempting
* to use the config. <code>checkValid()</code> is limited to reporting
* generic, but common, problems such as missing settings and blatant type
* incompatibilities.
*
* @param reference
* a reference configuration
* @param restrictToPaths
* only validate values underneath these paths that your code
* module owns and understands
* @throws ConfigException.ValidationFailed
* if there are any validation issues
* @throws ConfigException.NotResolved
* if this config is not resolved
* @throws ConfigException.BugOrBroken
* if the reference config is unresolved or caller otherwise
* misuses the API
*/
void checkValid(Config reference, String... restrictToPaths);
/**
* Checks whether a value is present and non-null at the given path. This
* differs in two ways from {@code Map.containsKey()} as implemented by

65
config/src/main/java/com/typesafe/config/ConfigException.java
View File

@ -3,6 +3,7 @@
*/
package com.typesafe.config;
/**
* All exceptions thrown by the library are subclasses of ConfigException.
*/
@ -180,10 +181,11 @@ public class ConfigException extends RuntimeException {
}
/**
* Exception indicating that there's a bug in something or the runtime
* environment is broken. This exception should never be handled; instead,
* something should be fixed to keep the exception from occurring.
*
* Exception indicating that there's a bug in something (possibly the
* library itself) or the runtime environment is broken. This exception
* should never be handled; instead, something should be fixed to keep the
* exception from occurring. This exception can be thrown by any method in
* the library.
*/
public static class BugOrBroken extends ConfigException {
private static final long serialVersionUID = 1L;
@ -264,4 +266,59 @@ public class ConfigException extends RuntimeException {
this(message, null);
}
}
public static class ValidationProblem {
final private String path;
final private ConfigOrigin origin;
final private String problem;
public ValidationProblem(String path, ConfigOrigin origin, String problem) {
this.path = path;
this.origin = origin;
this.problem = problem;
}
public String path() {
return path;
}
public ConfigOrigin origin() {
return origin;
}
public String problem() {
return problem;
}
}
public static class ValidationFailed extends ConfigException {
private static final long serialVersionUID = 1L;
final private Iterable<ValidationProblem> problems;
public ValidationFailed(Iterable<ValidationProblem> problems) {
super(makeMessage(problems), null);
this.problems = problems;
}
public Iterable<ValidationProblem> problems() {
return problems;
}
private static String makeMessage(Iterable<ValidationProblem> problems) {
StringBuilder sb = new StringBuilder();
for (ValidationProblem p : problems) {
sb.append(p.origin().description());
sb.append(": ");
sb.append(p.path());
sb.append(": ");
sb.append(p.problem());
sb.append(", ");
}
sb.setLength(sb.length() - 2); // chop comma and space
return sb.toString();
}
}
}

10
config/src/main/java/com/typesafe/config/impl/AbstractConfigObject.java
View File

@ -61,19 +61,23 @@ abstract class AbstractConfigObject extends AbstractConfigValue implements
* (just returns null if path not found). Does however resolve the path, if
* resolver != null.
*/
protected ConfigValue peekPath(Path path, SubstitutionResolver resolver,
protected AbstractConfigValue peekPath(Path path, SubstitutionResolver resolver,
int depth, ConfigResolveOptions options) {
return peekPath(this, path, resolver, depth, options);
}
private static ConfigValue peekPath(AbstractConfigObject self, Path path,
AbstractConfigValue peekPath(Path path) {
return peekPath(this, path, null, 0, null);
}
private static AbstractConfigValue peekPath(AbstractConfigObject self, Path path,
SubstitutionResolver resolver, int depth,
ConfigResolveOptions options) {
String key = path.first();
Path next = path.remainder();
if (next == null) {
ConfigValue v = self.peek(key, resolver, depth, options);
AbstractConfigValue v = self.peek(key, resolver, depth, options);
return v;
} else {
// it's important to ONLY resolve substitutions here, not

172
config/src/main/java/com/typesafe/config/impl/SimpleConfig.java
View File

@ -625,4 +625,176 @@ class SimpleConfig implements Config {
"Could not parse size-in-bytes number '" + numberString + "'");
}
}
private AbstractConfigValue peekPath(Path path) {
return root().peekPath(path);
}
private static void addProblem(List<ConfigException.ValidationProblem> accumulator, Path path,
ConfigOrigin origin, String problem) {
accumulator.add(new ConfigException.ValidationProblem(path.render(), origin, problem));
}
private static String getDesc(ConfigValue refValue) {
if (refValue instanceof AbstractConfigObject) {
AbstractConfigObject obj = (AbstractConfigObject) refValue;
if (obj.isEmpty())
return "object";
else
return "object with keys " + obj.keySet();
} else if (refValue instanceof SimpleConfigList) {
return "list";
} else {
return refValue.valueType().name().toLowerCase();
}
}
private static void addMissing(List<ConfigException.ValidationProblem> accumulator,
ConfigValue refValue, Path path, ConfigOrigin origin) {
addProblem(accumulator, path, origin, "No setting at '" + path.render() + "', expecting: "
+ getDesc(refValue));
}
private static void addWrongType(List<ConfigException.ValidationProblem> accumulator,
ConfigValue refValue, AbstractConfigValue actual, Path path) {
addProblem(accumulator, path, actual.origin(), "Wrong value type at '" + path.render()
+ "', expecting: " + getDesc(refValue) + " but got: "
+ getDesc(actual));
}
private static boolean couldBeNull(AbstractConfigValue v) {
return DefaultTransformer.transform(v, ConfigValueType.NULL)
.valueType() == ConfigValueType.NULL;
}
private static boolean haveCompatibleTypes(ConfigValue reference, AbstractConfigValue value) {
if (couldBeNull((AbstractConfigValue) reference) || couldBeNull(value)) {
// we allow any setting to be null
return true;
} else if (reference instanceof AbstractConfigObject) {
if (value instanceof AbstractConfigObject) {
return true;
} else {
return false;
}
} else if (reference instanceof SimpleConfigList) {
if (value instanceof SimpleConfigList) {
return true;
} else {
return false;
}
} else if (reference instanceof ConfigString) {
// assume a string could be gotten as any non-collection type;
// allows things like getMilliseconds including domain-specific
// interpretations of strings
return true;
} else if (value instanceof ConfigString) {
// assume a string could be gotten as any non-collection type
return true;
} else {
if (reference.valueType() == value.valueType()) {
return true;
} else {
return false;
}
}
}
// path is null if we're at the root
private static void checkValidObject(Path path, AbstractConfigObject reference,
AbstractConfigObject value,
List<ConfigException.ValidationProblem> accumulator) {
for (Map.Entry<String, ConfigValue> entry : reference.entrySet()) {
String key = entry.getKey();
Path childPath;
if (path != null)
childPath = Path.newKey(key).prepend(path);
else
childPath = Path.newKey(key);
AbstractConfigValue v = value.get(key);
if (v == null) {
addMissing(accumulator, entry.getValue(), childPath, value.origin());
} else {
checkValid(childPath, entry.getValue(), v, accumulator);
}
}
}
private static void checkValid(Path path, ConfigValue reference, AbstractConfigValue value,
List<ConfigException.ValidationProblem> accumulator) {
// Unmergeable is supposed to be impossible to encounter in here
// because we check for resolve status up front.
if (haveCompatibleTypes(reference, value)) {
if (reference instanceof AbstractConfigObject && value instanceof AbstractConfigObject) {
checkValidObject(path, (AbstractConfigObject) reference,
(AbstractConfigObject) value, accumulator);
} else if (reference instanceof SimpleConfigList && value instanceof SimpleConfigList) {
SimpleConfigList listRef = (SimpleConfigList) reference;
SimpleConfigList listValue = (SimpleConfigList) value;
if (listRef.isEmpty() || listValue.isEmpty()) {
// can't verify type, leave alone
} else {
AbstractConfigValue refElement = listRef.get(0);
for (ConfigValue elem : listValue) {
AbstractConfigValue e = (AbstractConfigValue) elem;
if (!haveCompatibleTypes(refElement, e)) {
addProblem(accumulator, path, e.origin(), "List at '" + path.render()
+ "' contains wrong value type, expecting list of "
+ getDesc(refElement) + " but got element of type "
+ getDesc(e));
// don't add a problem for every last array element
break;
}
}
}
}
} else {
addWrongType(accumulator, reference, value, path);
}
}
@Override
public void checkValid(Config reference, String... restrictToPaths) {
SimpleConfig ref = (SimpleConfig) reference;
// unresolved reference config is a bug in the caller of checkValid
if (ref.root().resolveStatus() != ResolveStatus.RESOLVED)
throw new ConfigException.BugOrBroken(
"do not call checkValid() with an unresolved reference config, call Config.resolve()");
// unresolved config under validation is probably a bug in something,
// but our whole goal here is to check for bugs in this config, so
// BugOrBroken is not the appropriate exception.
if (root().resolveStatus() != ResolveStatus.RESOLVED)
throw new ConfigException.NotResolved(
"config has unresolved substitutions; must call Config.resolve()");
// Now we know that both reference and this config are resolved
List<ConfigException.ValidationProblem> problems = new ArrayList<ConfigException.ValidationProblem>();
if (restrictToPaths.length == 0) {
checkValidObject(null, ref.root(), root(), problems);
} else {
for (String p : restrictToPaths) {
Path path = Path.newPath(p);
AbstractConfigValue refValue = ref.peekPath(path);
if (refValue != null) {
AbstractConfigValue child = peekPath(path);
if (child != null) {
checkValid(path, refValue, child, problems);
} else {
addMissing(problems, refValue, path, origin());
}
}
}
}
if (!problems.isEmpty()) {
throw new ConfigException.ValidationFailed(problems);
}
}
}

30
config/src/test/resources/validate-invalid.conf Normal file
View File

@ -0,0 +1,30 @@
string1="a string"
string2=107
string3={ a : b }
string4=[]
int1=203
int2="foo"
int3={ q : s }
float1="the string"
float2=false
float3=[ 4, 5, 6 ]
bool1=709
bool2="string!"
bool3={}
null1=10000
null2="hello world"
null3=true
object1={ z : s }
object2=[]
object3=12345
array1=[1,2,"foo"]
array2=[7,8,9]
array3=[{ n : m }, 10]
array4=[42, 43]
array5=64
emptyArray1=[1,2,3]
emptyArray2=["a","b","c"]
a.b.c.d.e.f.g = 100
a.b.c.d.e.f.h = "foo"
a.b.c.d.e.f.i = []

32
config/src/test/resources/validate-reference.conf Normal file
View File

@ -0,0 +1,32 @@
string1="foo"
string2="bar"
string3="baz"
string4="hello"
int1=10
int2=11
int3=12
float1=3.14
float2=3.2
float3=3.3
bool1=true
bool2=false
bool3=true
null1=null
null2=null
null3=null
object1={ a : b }
object2={ c : d }
object3={ e : f }
array1=[1,2,3]
array2=[a,b,c]
array3=[true, true, false]
array4=[{}, {}]
array5=[]
emptyArray1=[]
emptyArray2=[]
willBeMissing=90009
a.b.c.d.e.f.g = true
a.b.c.d.e.f.h = true
a.b.c.d.e.f.i = true
a.b.c.d.e.f.j = true

122
config/src/test/scala/com/typesafe/config/impl/ValidationTest.scala Normal file
View File

@ -0,0 +1,122 @@
/**
* Copyright (C) 2011 Typesafe Inc. <http://typesafe.com>
*/
package com.typesafe.config.impl
import org.junit.Assert._
import org.junit._
import com.typesafe.config.ConfigFactory
import com.typesafe.config.ConfigParseOptions
import com.typesafe.config.ConfigException
import scala.collection.JavaConverters._
import scala.io.Source
class ValidationTest extends TestUtils {
sealed abstract class Problem(path: String, line: Int) {
def check(p: ConfigException.ValidationProblem) {
assertEquals(path, p.path())
assertEquals(line, p.origin().lineNumber())
}
protected def assertMessage(p: ConfigException.ValidationProblem, re: String) {
assertTrue("didn't get expected message for " + path + ": got '" + p.problem() + "'",
p.problem().matches(re))
}
}
case class Missing(path: String, line: Int, expected: String) extends Problem(path, line) {
override def check(p: ConfigException.ValidationProblem) {
super.check(p)
val re = "No setting.*" + path + ".*expecting.*" + expected + ".*"
assertMessage(p, re)
}
}
case class WrongType(path: String, line: Int, expected: String, got: String) extends Problem(path, line) {
override def check(p: ConfigException.ValidationProblem) {
super.check(p)
val re = "Wrong value type.*" + path + ".*expecting.*" + expected + ".*got.*" + got + ".*"
assertMessage(p, re)
}
}
case class WrongElementType(path: String, line: Int, expected: String, got: String) extends Problem(path, line) {
override def check(p: ConfigException.ValidationProblem) {
super.check(p)
val re = "List at.*" + path + ".*wrong value type.*expecting.*" + expected + ".*got.*element of.*" + got + ".*"
assertMessage(p, re)
}
}
private def checkException(e: ConfigException.ValidationFailed, expecteds: Seq[Problem]) {
val problems = e.problems().asScala.toIndexedSeq[ConfigException.ValidationProblem]
.sortBy(_.path).sortBy(_.origin.lineNumber)
//for (problem <- problems)
// System.err.println(problem.origin().description() + ": " + problem.path() + ": " + problem.problem())
for ((problem, expected) <- problems zip expecteds) {
expected.check(problem)
}
assertEquals(expecteds.size, problems.size)
}
@Test
def validation() {
val reference = ConfigFactory.parseFile(resourceFile("validate-reference.conf"), ConfigParseOptions.defaults())
val conf = ConfigFactory.parseFile(resourceFile("validate-invalid.conf"), ConfigParseOptions.defaults())
val e = intercept[ConfigException.ValidationFailed] {
conf.checkValid(reference)
}
val expecteds = Seq(
Missing("willBeMissing", 1, "number"),
WrongType("int3", 7, "number", "object"),
WrongType("float2", 9, "number", "boolean"),
WrongType("float3", 10, "number", "list"),
WrongType("bool1", 11, "boolean", "number"),
WrongType("bool3", 13, "boolean", "object"),
Missing("object1.a", 17, "string"),
WrongType("object2", 18, "object", "list"),
WrongType("object3", 19, "object", "number"),
WrongElementType("array3", 22, "boolean", "object"),
WrongElementType("array4", 23, "object", "number"),
WrongType("array5", 24, "list", "number"),
WrongType("a.b.c.d.e.f.g", 28, "boolean", "number"),
Missing("a.b.c.d.e.f.j", 28, "boolean"),
WrongType("a.b.c.d.e.f.i", 30, "boolean", "list"))
checkException(e, expecteds)
}
@Test
def validationWithRoot() {
val objectWithB = parseObject("""{ b : c }""")
val reference = ConfigFactory.parseFile(resourceFile("validate-reference.conf"),
ConfigParseOptions.defaults()).withFallback(objectWithB)
val conf = ConfigFactory.parseFile(resourceFile("validate-invalid.conf"), ConfigParseOptions.defaults())
val e = intercept[ConfigException.ValidationFailed] {
conf.checkValid(reference, "a", "b")
}
val expecteds = Seq(
Missing("b", 1, "string"),
WrongType("a.b.c.d.e.f.g", 28, "boolean", "number"),
Missing("a.b.c.d.e.f.j", 28, "boolean"),
WrongType("a.b.c.d.e.f.i", 30, "boolean", "list"))
checkException(e, expecteds)
}
@Test
def validationCatchesUnresolved() {
val reference = parseConfig("""{ a : 2 }""")
val conf = parseConfig("""{ b : ${c}, c : 42 }""")
val e = intercept[ConfigException.NotResolved] {
conf.checkValid(reference)
}
assertTrue("expected different message, got: " + e.getMessage,
e.getMessage.contains("unresolved"))
}
}