12: what about cycles?

(NOTE: this section digs into some intricate details of how Specmonstah works. It might be confusing and it could use improvement.)
Let's say that in your forum all posts reference a parent topic, and that topics reference their first post. That means that for every topic, its :first-post-id that should be set to a post's :id, and that's post's :topic-id must reference the topic's :id. What we got here is a good ol' fashioned cycle:
1
(ns reifyhealth.specmonstah-tutorial.12
2
  (:require [clojure.spec.alpha :as s]
3
            [clojure.spec.gen.alpha :as gen]
4
            [reifyhealth.specmonstah.core :as sm]
5
            [reifyhealth.specmonstah.spec-gen :as sg]))
6
​
7
(def id-seq (atom 0))
8
;; This spec uses a generated such that each time a value is generated
9
;; it returns the incremented value of the atom id-seq
10
(s/def ::id (s/with-gen pos-int?
11
              #(gen/fmap (fn [_] (swap! id-seq inc))
12
                         (gen/return nil))))
13
(s/def ::topic-id ::id)
14
(s/def ::first-post-id ::id)
15
​
16
(s/def ::post (s/keys :req-un [::id ::topic-id]))
17
(s/def ::topic (s/keys :req-un [::id ::first-post-id]))
18
​
19
(def schema
20
  {:topic {:prefix    :t
21
           :spec      ::topic
22
           :relations {:first-post-id [:post :id]}
23
           :conform   {:cycle-keys #{:first-post-id}}}
24
   :post  {:prefix      :p
25
           :relations   {:topic-id [:topic :id]}
26
           :constraints {:topic-id #{:required}}
27
           :spec        ::post}})
28
​
29
(defn ex-01
30
  []
31
  (sm/view (sm/add-ents {:schema schema} {:post [[3]]})))
32
  
33
(ex-01)
Copied!
behold! a cycle!
This graph shows how the ents are related: :p0, :p1, and :p2 all reference the topic :t0, and :t0 references :p0.
One problem that could arise in this situation is that your database might have a foreign key constraint on the post table requiring that a post's :topic-id be NOT NULL and that it reference a topic that actually exists. The topic's :first-post-id might allow NULL values, but have the constraint that the field reference a post that actually exist. However, when cycles exist in a graph there's no way to know which of the two nodes in the cycle should come first. How do we make sure that Specmonstah does the following?
1.
Inserts a :topic without a value for :first-post-id 
2.
Inserts a :post that references the :topic 
3.
Performs an update on the :topic to set it's :first-post-id to the :post's :id
There are two parts to solving this problem. The first is to ensure that ents are sorted correctly for visiting functions. Since you can't topologically sort a cycle, we need some way to tell Specmonstah how to get from a graph with a cycle to a graph without one. We do that with the :required constraint. Have another look at the schema:
1
(def schema
2
  {:topic {:prefix    :t
3
           :spec      ::topic
4
           :relations {:first-post-id [:post :id]}
5
           :conform   {:cycle-keys #{:first-post-id}}}
6
   :post  {:prefix      :p
7
           :relations   {:topic-id [:topic :id]}
8
           :constraints {:topic-id #{:required}}
9
           :spec        ::post}})
Copied!
You can see that the the :post definition includes :constraints {:topic-id #{:required}}. This is how you tell specmonstah, "Make sure that the :topic that this :topic-id refers to gets visited before this :post". Internally, this instructs Specmonstah to create a temporary graph where the directed edges from topics to posts are removed, and then topologically sort that graph. You can see it at work:
1
(defn ex-02
2
  "Shows that `:required` results in correct order"
3
  []
4
  (-> (sm/add-ents {:schema schema} {:post [[3]]})
5
      (sm/visit-ents :print (fn [_ {:keys [ent-name]}] (prn ent-name))))
6
  nil)
7
(ex-02)
8
=>
9
:t0
10
:p2
11
:p1
12
:p0
Copied!
The second part to solving this problem is to break your visiting function up into multiple functions:
1
(def database (atom []))
2
​
3
(defn insert
4
  "When inserting records, remove any `:cycle-keys` because those keys
5
  will reference records that haven't been inserted yet."
6
  [db {:keys [ent-name ent-type visit-query-opts spec-gen schema-opts]}]
7
  (let [cycle-keys (into (:cycle-keys schema-opts) (:cycle-keys visit-query-opts))
8
        record     (apply dissoc spec-gen cycle-keys)]
9
    (swap! database conj [:insert ent-name record])))
10
​
11
(defn update-keys
12
  "Perform an 'update', setting all cycle keys to the correct value now
13
  that the referenced record exists"
14
  [db {:keys [ent-name ent-type visit-query-opts spec-gen schema-opts]}]
15
  (let [cycle-keys (into (:cycle-keys schema-opts) (:cycle-keys visit-query-opts))]
16
    (when (seq cycle-keys)
17
      (swap! database conj [:update ent-name (select-keys spec-gen cycle-keys)]))))
18
​
19
(def conform [insert update-keys])
20
​
21
(defn ex-03
22
  []
23
  (reset! database [])
24
  (reset! id-seq 0)
25
  (-> (sg/ent-db-spec-gen {:schema schema} {:post [[3]]})
26
      (sm/visit-ents :conform conform))
27
  @database)
28
​
29
(ex-03) ; =>
30
[[:insert :t0 {:id 1}]
31
 [:insert :p2 {:id 3, :topic-id 1}]
32
 [:insert :p1 {:id 5, :topic-id 1}]
33
 [:insert :p0 {:id 7, :topic-id 1}]
34
 [:update :t0 {:first-post-id 7}]]
Copied!
Here we're creating a visiting "function" named conform which is actually a vector of functions. The insert function is applied to all ents in topsort order, then the update-keys function is applied to ents in topsort order. You can see that :t0 is "inserted" without a :first-post-id, and then after all records have been inserted :t0 is "updated", setting its :first-post-id.
This introduces one more question: How does the visiting function know to leave out :first-post-id when the insert function is applied, and add it when the update-keys function is applied?
The answer has multiple parts. First, let's look at the schema again:
1
(def schema
2
  {:topic {:prefix    :t
3
           :spec      ::topic
4
           :relations {:first-post-id [:post :id]}
5
           :conform   {:cycle-keys #{:first-post-id}}}
6
   :post  {:prefix      :p
7
           :relations   {:topic-id [:topic :id]}
8
           :constraints {:topic-id #{:required}}
9
           :spec        ::post}})
Copied!
Notice the :conform key in :topic schema. The :conform key doesn't have any special meaning; we name it to match the visiting key for the conform visiting function. When the conform visiting function is applied, the value of the :conform key is passed in under the :schema-opts key. You can see that the insert function uses this:
1
(defn insert
2
  "When inserting records, remove any `:cycle-keys` because those keys
3
  will reference records that haven't been inserted yet."
4
  [db {:keys [ent-name ent-type visit-query-opts spec-gen schema-opts]}]
5
  (let [cycle-keys (into (:cycle-keys schema-opts) (:cycle-keys visit-query-opts))
6
        record     (apply dissoc spec-gen cycle-keys)]
7
    (swap! database conj [:insert ent-name record])))
Copied!
The insert function creates the record to be inserted by dissocing any keys found in :cycle-keys of schema-opts.
11: collection constraint (vector of foreign keys)
Last modified 2yr ago
Copy link