Human Factors of XR: Using Human Factors to Design XR Systems
UI Design From Scratch - Part 5 - transcript.pdf
1. UI Design From Scratch - Part V
After going over the CSS it’s time to go into the code changes we made for this design
2. public class Dish implements PropertyBusinessObject {
public final Property<String, Dish> id = new Property<>("id");
public final DoubleProperty<Dish> price = new DoubleProperty<>("price");
public final Property<String, Dish> name = new Property<>("name");
public final Property<String, Dish> description = new Property<>("description");
public final Property<String, Dish> imageName = new Property<>("imageName");
public final Property<Image, Dish> thumbnail = new Property<Image, Dish>("thumbnail") {
public Image get() {
// snipped common code
}
};
public final Property<Image, Dish> fullSize = new Property<Image, Dish>("fullSize") {
public Image get() {
Image i = super.get();
if(i == null) {
i = fetchImage();
super.set(i);
}
return i;
}
};
private final PropertyIndex idx = new PropertyIndex(this, "Dish",
id, price, name, description, imageName, thumbnail, fullSize);
public PropertyIndex getPropertyIndex() {
return idx;
}
}
Dish
First we need some common business logic to represent a “Dish”. I used the properties API to represent the dish object. A dish contains a unique id, price, name,
description and image in two sizes: thumbnail & full size.
3. public class AboutForm extends BaseForm {
public AboutForm() {
super("About");
}
@Override
protected Container createContent() {
BrowserComponent cmp = new BrowserComponent();
try {
cmp.setURLHierarchy("/index.html");
} catch(IOException err) {
Log.e(err);
Log.sendLog();
}
return cmp;
}
@Override
protected void onSearch(String searchString) {
}
}
About
The about form is pretty trivial. We store the full HTML hierarchy under the HTML directory which acts as a special case and allows us to view HTML with all the bells and
whistles on the device
4. Container topArea = FlowLayout.encloseRight(createCartButton());
topArea.setUIID("TopBar");
tb.addComponentToSideMenu(topArea);
tb.addMaterialCommandToSideMenu("Menu",
FontImage.MATERIAL_RESTAURANT_MENU,
e -> new MainMenuForm().show());
tb.addMaterialCommandToSideMenu("About",
FontImage.MATERIAL_INFO,
e -> new AboutForm().show());
tb.addMaterialCommandToSideMenu("Contact Us",
FontImage.MATERIAL_MAP,
e -> new ContactUsForm().show());
//….
Component.setSameHeight(topArea, topBar);
Base Form
We use enclose layout to position the cart icon on the top right. This encloses the button in a flow layout to position it in the right place. The top area is big enough to
contain the image we see here and its size is determined by the size of the topBar which is the top of the main form. However because we use encloseRight the cart
button is positioned correctly
5. final static String[] PICKER_STRINGS = new String[100];
static {
PICKER_STRINGS[0] = "Delete";
for(int iter = 1 ; iter < 100 ; iter++) {
PICKER_STRINGS[iter] = iter + "x";
}
}
Picker quantityButton = new Picker();
quantityButton.setUIID("QuantityPicker");
quantityButton.setType(Display.PICKER_TYPE_STRINGS);
quantityButton.setStrings(PICKER_STRINGS);
quantityButton.setSelectedString(quantity + "x");
Container dishContainer = BoxLayout.encloseX(
FlowLayout.encloseMiddle(quantityButton),
new Label(dishMasked, "UnmarginedLabel"),
FlowLayout.encloseMiddle(
BoxLayout.encloseY(
new Label(title, "DishCheckoutTitle"),
new Label(L10NManager.getInstance().formatCurrency(price), "CheckoutPrice")
)
)
);
Checkout Form
The picker component is just a hardcoded set of values from 1 to 100 with a special case for zero. We use a picker class to represent that but the layout of the main
widget is interesting here…
6. final static String[] PICKER_STRINGS = new String[100];
static {
PICKER_STRINGS[0] = "Delete";
for(int iter = 1 ; iter < 100 ; iter++) {
PICKER_STRINGS[iter] = iter + "x";
}
}
Picker quantityButton = new Picker();
quantityButton.setUIID("QuantityPicker");
quantityButton.setType(Display.PICKER_TYPE_STRINGS);
quantityButton.setStrings(PICKER_STRINGS);
quantityButton.setSelectedString(quantity + "x");
Container dishContainer = BoxLayout.encloseX(
FlowLayout.encloseMiddle(quantityButton),
new Label(dishMasked, "UnmarginedLabel"),
FlowLayout.encloseMiddle(
BoxLayout.encloseY(
new Label(title, "DishCheckoutTitle"),
new Label(L10NManager.getInstance().formatCurrency(price), "CheckoutPrice")
)
)
);
Checkout Form
We have a box layout X for the elements which arranges them in a row. It’s important not to use flow layout for the entire row as it might break a line which isn’t
something we want at this point. However, if we use flow layout for a single element breaking a line isn’t an option so this won’t be a problem.
7. final static String[] PICKER_STRINGS = new String[100];
static {
PICKER_STRINGS[0] = "Delete";
for(int iter = 1 ; iter < 100 ; iter++) {
PICKER_STRINGS[iter] = iter + "x";
}
}
Picker quantityButton = new Picker();
quantityButton.setUIID("QuantityPicker");
quantityButton.setType(Display.PICKER_TYPE_STRINGS);
quantityButton.setStrings(PICKER_STRINGS);
quantityButton.setSelectedString(quantity + "x");
Container dishContainer = BoxLayout.encloseX(
FlowLayout.encloseMiddle(quantityButton),
new Label(dishMasked, "UnmarginedLabel"),
FlowLayout.encloseMiddle(
BoxLayout.encloseY(
new Label(title, "DishCheckoutTitle"),
new Label(L10NManager.getInstance().formatCurrency(price), "CheckoutPrice")
)
)
);
Checkout Form
We wrap the quantity button in a flow layout so it will be in the middle we do the same for the two labels. Since both comprise of one item the flakyness that is
sometimes associated with flow layout isn’t applicable and we can just use it as a trick to center align elements
8. final static String[] PICKER_STRINGS = new String[100];
static {
PICKER_STRINGS[0] = "Delete";
for(int iter = 1 ; iter < 100 ; iter++) {
PICKER_STRINGS[iter] = iter + "x";
}
}
Picker quantityButton = new Picker();
quantityButton.setUIID("QuantityPicker");
quantityButton.setType(Display.PICKER_TYPE_STRINGS);
quantityButton.setStrings(PICKER_STRINGS);
quantityButton.setSelectedString(quantity + "x");
Container dishContainer = BoxLayout.encloseX(
FlowLayout.encloseMiddle(quantityButton),
new Label(dishMasked, "UnmarginedLabel"),
FlowLayout.encloseMiddle(
BoxLayout.encloseY(
new Label(title, "DishCheckoutTitle"),
new Label(L10NManager.getInstance().formatCurrency(price), "CheckoutPrice")
)
)
);
Checkout Form
We don’t need to center align the image because it is physically larger than the other elements
9. final static String[] PICKER_STRINGS = new String[100];
static {
PICKER_STRINGS[0] = "Delete";
for(int iter = 1 ; iter < 100 ; iter++) {
PICKER_STRINGS[iter] = iter + "x";
}
}
Picker quantityButton = new Picker();
quantityButton.setUIID("QuantityPicker");
quantityButton.setType(Display.PICKER_TYPE_STRINGS);
quantityButton.setStrings(PICKER_STRINGS);
quantityButton.setSelectedString(quantity + "x");
Container dishContainer = BoxLayout.encloseX(
FlowLayout.encloseMiddle(quantityButton),
new Label(dishMasked, "UnmarginedLabel"),
FlowLayout.encloseMiddle(
BoxLayout.encloseY(
new Label(title, "DishCheckoutTitle"),
new Label(L10NManager.getInstance().formatCurrency(price), "CheckoutPrice")
)
)
);
Checkout Form
The labels are arranged as a box layout Y, notice they are in the middle because of the flow layout surrounding the box Y
10. quantityButton.addActionListener(e -> {
if(quantityButton.getSelectedString().equals(PICKER_STRINGS[0])) {
Display.getInstance().callSerially(() -> {
dishContainer.setX(Display.getInstance().getDisplayWidth());
Container p = dishContainer.getParent();
p.animateUnlayoutAndWait(250, 255);
dishContainer.remove();
p.animateLayout(200);
});
}
});
Checkout Form
The quantity button allows us to edit the number of elements and also remove an element from the list. If you look at the removal effect looping to the right here you will
notice that this animation has several stages.
First we set the dish to an X position outside of the form and call animate unlayout to show the floating out effect. Animate unlayout moves a layout from a valid laid out
state to an invalid state and is designed exactly for this use case. Next we physically remove the element which is now outside of the form and we animate the UI layout.
This creates this cool layout effect
11. Coord crd = new Coord(40.732030, -73.999872);
MapContainer map = new MapContainer(“key-from-google“);
map.addMarker(null, crd, "Blue Hill", "Description", null);
map.setCameraPosition(crd);
TextArea address = new TextArea("Blue Hill,n"
+ "Farm-to-table American fine diningn"
+ "75 Washington Pln"
+ "New York, NY 10011n"
+ "+1 212-539-1776");
address.setEditable(false);
address.setUIID("MapAddressText");
Button phone = new Button("", "ShoppingCart");
FontImage.setMaterialIcon(phone, FontImage.MATERIAL_CALL);
phone.addActionListener(e -> Display.getInstance().dial("+12125391776"));
Button navigate = new Button("", "ShoppingCart");
FontImage.setMaterialIcon(navigate, FontImage.MATERIAL_NAVIGATION);
navigate.addActionListener(e ->
Display.getInstance().openNativeNavigationApp(address));
Container addressContainer = BorderLayout.center(address);
addressContainer.add(BorderLayout.EAST,
GridLayout.encloseIn(1, phone, navigate));
addressContainer.setUIID("MapAddress");
Container lp = LayeredLayout.encloseIn(map,
BorderLayout.south(addressContainer));
return lp;
Contact Us
The contact us form is relatively simple, it has a lot of data in it but all of that is pretty standard. First we have the MapContainer class which is a part of the Codename
One Google Maps cn1lib that we installed into the application. This can be installed via the extensions menu and allows us to place a native Map on the form. The
coordinates of the map are all hardcoded at this point
12. Coord crd = new Coord(40.732030, -73.999872);
MapContainer map = new MapContainer(“key-from-google“);
map.addMarker(null, crd, "Blue Hill", "Description", null);
map.setCameraPosition(crd);
TextArea address = new TextArea("Blue Hill,n"
+ "Farm-to-table American fine diningn"
+ "75 Washington Pln"
+ "New York, NY 10011n"
+ "+1 212-539-1776");
address.setEditable(false);
address.setUIID("MapAddressText");
Button phone = new Button("", "ShoppingCart");
FontImage.setMaterialIcon(phone, FontImage.MATERIAL_CALL);
phone.addActionListener(e -> Display.getInstance().dial("+12125391776"));
Button navigate = new Button("", "ShoppingCart");
FontImage.setMaterialIcon(navigate, FontImage.MATERIAL_NAVIGATION);
navigate.addActionListener(e ->
Display.getInstance().openNativeNavigationApp(address));
Container addressContainer = BorderLayout.center(address);
addressContainer.add(BorderLayout.EAST,
GridLayout.encloseIn(1, phone, navigate));
addressContainer.setUIID("MapAddress");
Container lp = LayeredLayout.encloseIn(map,
BorderLayout.south(addressContainer));
return lp;
Contact Us
The address is just a text area that isn’t editable styled to appear in this specific way
13. Coord crd = new Coord(40.732030, -73.999872);
MapContainer map = new MapContainer(“key-from-google“);
map.addMarker(null, crd, "Blue Hill", "Description", null);
map.setCameraPosition(crd);
TextArea address = new TextArea("Blue Hill,n"
+ "Farm-to-table American fine diningn"
+ "75 Washington Pln"
+ "New York, NY 10011n"
+ "+1 212-539-1776");
address.setEditable(false);
address.setUIID("MapAddressText");
Button phone = new Button("", "ShoppingCart");
FontImage.setMaterialIcon(phone, FontImage.MATERIAL_CALL);
phone.addActionListener(e -> Display.getInstance().dial("+12125391776"));
Button navigate = new Button("", "ShoppingCart");
FontImage.setMaterialIcon(navigate, FontImage.MATERIAL_NAVIGATION);
navigate.addActionListener(e ->
Display.getInstance().openNativeNavigationApp(address));
Container addressContainer = BorderLayout.center(address);
addressContainer.add(BorderLayout.EAST,
GridLayout.encloseIn(1, phone, navigate));
addressContainer.setUIID("MapAddress");
Container lp = LayeredLayout.encloseIn(map,
BorderLayout.south(addressContainer));
return lp;
Contact Us
The Phone and Navigate buttons are just standard Codename One buttons with the shopping cart style. They map to the appropriate native functionality in the Display
class.
14. Coord crd = new Coord(40.732030, -73.999872);
MapContainer map = new MapContainer(“key-from-google“);
map.addMarker(null, crd, "Blue Hill", "Description", null);
map.setCameraPosition(crd);
TextArea address = new TextArea("Blue Hill,n"
+ "Farm-to-table American fine diningn"
+ "75 Washington Pln"
+ "New York, NY 10011n"
+ "+1 212-539-1776");
address.setEditable(false);
address.setUIID("MapAddressText");
Button phone = new Button("", "ShoppingCart");
FontImage.setMaterialIcon(phone, FontImage.MATERIAL_CALL);
phone.addActionListener(e -> Display.getInstance().dial("+12125391776"));
Button navigate = new Button("", "ShoppingCart");
FontImage.setMaterialIcon(navigate, FontImage.MATERIAL_NAVIGATION);
navigate.addActionListener(e ->
Display.getInstance().openNativeNavigationApp(address));
Container addressContainer = BorderLayout.center(address);
addressContainer.add(BorderLayout.EAST,
GridLayout.encloseIn(1, phone, navigate));
addressContainer.setUIID("MapAddress");
Container lp = LayeredLayout.encloseIn(map,
BorderLayout.south(addressContainer));
return lp;
Contact Us
We overlay the text and buttons container on top of the map using the layered layout. Since the map is a native component there are implications to placing an overlay on
top but there are a bit too much for this discussion. If you are interested in learning more about that check out the developer guide sections on peer components
15. super(actualDish.name.get(), new LayeredLayout());
ImageViewer iv = new ImageViewer();
iv.setImage(actualDish.fullSize.get());
add(iv);
iv.setImageInitialPosition(ImageViewer.IMAGE_FILL);
Toolbar tb = getToolbar();
tb.setUIID("TintToolbar");
Form previous = Display.getInstance().getCurrent();
tb.addMaterialCommandToLeftBar("",
FontImage.MATERIAL_CLOSE, ev -> previous.showBack());
tb.addMaterialCommandToRightBar("",
FontImage.MATERIAL_ADD_SHOPPING_CART, ev -> {});
TextArea description = new TextArea(actualDish.description.get());
description.setEditable(false);
description.setUIID("DishViewDescription");
add(BorderLayout.south(description));
Dish View
The dish view is very similar to the map view in regard to the bottom description and the overlay.
Notice that I went with an ImageViewer instead of using an image. This allows pinch to zoom that will even work with the overlay on top and will allow us to look more
closely at the dishes.
16. Label priceLabel = new Label(L10NManager.getInstance().
formatCurrency(actualDish.price.get()), "YourOrder");
priceLabel.getUnselectedStyle().setPaddingRight(6);
Image priceImage = Image.createImage(priceLabel.getPreferredW() -
Display.getInstance().convertToPixels(4),
priceLabel.getPreferredH(), 0);
priceLabel.setWidth(priceLabel.getPreferredW());
priceLabel.setHeight(priceLabel.getPreferredH());
priceLabel.paintComponent(priceImage.getGraphics(), false);
setGlassPane((g, rect) -> {
g.drawImage(priceImage,
getWidth() - priceImage.getWidth(), getHeight() / 5);
});
Dish View
The price label is essentially the same “YourOrder” component that appears in the top of the standard forms. However, if we tried to draw it “cut off” the round border
wouldn’t let us. The layout manager prevents us from placing a component “offscreen” as well. There are other hacks we could do but I chose to just paint the
component onto an image that is too small to hold the full size of the component then place that image on the screen using a glass pane. A glass pane is drawn as an
overlay on top of the entire screen and you can draw anything there.
17. What did we learn?
✦ Design has common concepts with programming
but a different set of rules
✦ We can’t replace a designer but we can fill in
gaps
✦ Let existing design do the talking: Images, Maps
etc. should dominate the UI
There are surprising parallels between best practices in almost every discipline I ever learned. Design is no different, you can take a lot of the core tenets of programming
and apply them to design once you understand the basic philosophy.
If you look at the previous UI elements it’s pretty obvious which ones were designed by a designer and which ones weren’t. But if we showed them to someone who isn’t
sensitive to design nuance it might take some guesses so we can get by but not much more than that.
When in doubt I take the lazy approach, I just use images or other elements to cover up the lack of design skills.
18. Thank You!
Thanks for watching I hope you found this educational. Please join the discussion in the comments section and let me know what you think!